Add ICNet for image segmentation. #975

4luojing · 2018-06-11T01:18:36Z

No description provided.

wanghaoshuang · 2018-06-12T08:47:00Z

fluid/icnet/cityscape.py

+import numpy as np
+import paddle.v2 as paddle
+
+DATA_PATH = "../../data/cityscape"


默认路径修改为"./data/cityscape"

wanghaoshuang · 2018-06-12T08:48:12Z

fluid/icnet/cityscape.py

+                yield image, label
+        return reader
+
+def load(image_label):


每个方法都要给出简要说明。

wanghaoshuang · 2018-06-12T08:48:53Z

fluid/icnet/cityscape.py

+    return image_labels[0].astype("float32"), label_sub1,  mask_sub1.astype("int32"), label_sub2, mask_sub2.astype("int32"), label_sub4, mask_sub4.astype("int32")
+
+
+def train(batch_size=32, random_mirror=False, random_scaling=False):


对外暴露的方法要给出详细使用说明。

wanghaoshuang · 2018-06-12T08:51:04Z

fluid/icnet/infer.py

+    _, _, sub124_out = network.icnet(args, images, num_classes, np.array(data_shape[1:]).astype("float32"), is_test=True)
+    predict = fluid.layers.upsampling_bilinear2d(sub124_out, out_shape=data_shape[1:3])
+    predict = fluid.layers.transpose(predict, perm=[0,2,3,1])
+    fluid.layers.Print(predict, summarize=10)


去掉Print语句

wanghaoshuang · 2018-06-12T08:54:34Z

fluid/icnet/network.py

+import sys
+
+
+def conv(input, k_h, k_w, c_o, s_h, s_w, relu=True, padding="VALID", group=1, biased=None, name=None, print_pad=False):


如果group没有被用到，建议将其从参数列表删除。

wanghaoshuang · 2018-06-12T08:55:44Z

fluid/icnet/network.py

+import sys
+
+
+def conv(input, k_h, k_w, c_o, s_h, s_w, relu=True, padding="VALID", group=1, biased=None, name=None, print_pad=False):


print_op 貌似也是多余的。

wanghaoshuang · 2018-06-12T08:57:10Z

fluid/icnet/network.py

+
+def interp(input, out_shape):
+    out_shape = list(out_shape.astype("int32"))
+    return fluid.layers.upsampling_bilinear2d(input, out_shape=out_shape)


最新paddle api已经将upsampling_bilinear2d 重命名为resize_bilinear

wanghaoshuang · 2018-06-12T08:57:51Z

fluid/icnet/network.py

+    conv3_3_1_1_increase = conv(conv3_3_3_3_bn, 1, 1, 256, 1, 1, biased=False, relu=False, name="conv3_3_1_1_increase")
+    conv3_3_1_1_increase_bn = bn(conv3_3_1_1_increase, relu=False, name="conv3_3_1_1_increase_bn", is_test=is_test)
+
+


空行太多。

lgone2000 · 2018-06-14T01:39:41Z

能否在image-segmentatation 建立一个子目录?便于未来扩展? by guoyi

qingqing01 · 2018-06-14T09:43:58Z

@lgone2000 good idea.

wanghaoshuang · 2018-06-14T07:23:28Z

fluid/icnet/README.md

+
+
+
+本文采用Cityscape数据集，请前往[Cityscape官网]()注册下载。下载数据之后，按照[这里](https://github.com/mcordts/cityscapesScripts)的说明和工具处理数据。


官网地址补充完整。

wanghaoshuang · 2018-06-14T07:28:25Z

fluid/icnet/eval.py

+        place = fluid.CUDAPlace(0)
+    exe = fluid.Executor(place)
+    exe.run(fluid.default_startup_program())
+    fluid.io.load_persistables(exe, args.model_path)


对于evaluator来说，只load parameter就行。

wanghaoshuang · 2018-06-14T07:28:55Z

fluid/icnet/eval.py

+    exe = fluid.Executor(place)
+    exe.run(fluid.default_startup_program())
+    fluid.io.load_persistables(exe, args.model_path)
+    print "loaded model from: %s" % args.model_path


判断下model_path是否存在。

wanghaoshuang · 2018-06-14T07:29:53Z

fluid/icnet/eval.py

+                             fetch_list=fetch_vars)
+        out_wrong += result[1]
+        out_right += result[2]
+        print "count: %s; current iou: %.3f;" % (count, result[0])


建议在同行输出。

wanghaoshuang · 2018-06-14T07:31:00Z

fluid/icnet/eval.py

+add_arg('use_gpu',           bool,  True,       "Whether use GPU to test.")
+# yapf: enable
+
+def cal_mean_iou(wrong, cerroct, num_classes):


cerroct -> correct

wanghaoshuang · 2018-06-14T07:34:50Z

fluid/icnet/eval.py

+
+def cal_mean_iou(wrong, cerroct, num_classes):
+    sum = wrong + cerroct
+    return (cerroct.astype("float64") / sum).sum() / num_classes


如果class i 从来没出现过，那么sum[i]就是0, 不能直接除；最后求平均也要除真实出现的class数量。

另外num_classes == len(num_classes), 所以可以去掉num_classes这个参数。

wanghaoshuang · 2018-06-14T07:37:07Z

fluid/icnet/infer.py

+add_arg('model_path',        str,   "",         "Model path.")
+add_arg('images_list',       str,   "",         "List file with images to be infered.")
+add_arg('images_path',       str,   "",         "The images path.")
+add_arg('out_path',          str,   "",         "Output path.")


给个默认的out_path. 其它路径也给个默认值。

wanghaoshuang · 2018-06-14T07:37:58Z

fluid/icnet/infer.py

+                ,[119, 10, 32]]
+                # 18 = bicycle
+
+def color(input):


说明下方法的作用

wanghaoshuang · 2018-06-14T07:39:10Z

fluid/icnet/infer.py

+        place = fluid.CUDAPlace(0)
+    exe = fluid.Executor(place)
+    exe.run(fluid.default_startup_program())
+    fluid.io.load_persistables(exe, args.model_path)


load parameter既可

wanghaoshuang · 2018-06-14T07:40:10Z

fluid/icnet/infer.py

+        image_t = fluid.core.LoDTensor()
+        image_t.set(img, place) 
+        result = exe.run(feed={"image": image_t}, fetch_list=[predict])
+        cv2.imwrite(args.out_path + "/" + filename + "_result.png", color(result[0]))


如果args.out_path不存在，需要新建一个。

qingqing01 · 2018-06-14T09:47:24Z

fluid/icnet/README.md

+- https://github.com/hszhao/ICNet
+- https://github.com/hellochick/ICNet-tensorflow
+- https://github.com/mcordts/cityscapesScripts
+- https://zhuanlan.zhihu.com/p/26653218


参考文献给知乎不够严谨，只保留一篇paper就行了？

qingqing01 · 2018-06-14T09:48:01Z

fluid/icnet/README.md

+<p align="center">
+<img src="images/train_loss.png" width="620" hspace='10'/> <br/>
+<strong>图 2</strong>
+</p>


有mean IoU的结果吗？

qingqing01 · 2018-06-14T09:49:44Z

fluid/icnet/README.md

+
+## 简介
+
+Image Cascade Network（ICNet），在兼顾实时性的同时，比原来的Fast Semantic Segmentation,比如SQ, SegNet, ENet等大大地提高了准确率，足以与Deeplab v2媲美，给语义分割的落地提供了可能。


这段中文和后面链接里知乎的中文相同，需要修改。

qingqing01 · 2018-06-14T09:50:26Z

fluid/icnet/README.md

+## 简介
+
+Image Cascade Network（ICNet），在兼顾实时性的同时，比原来的Fast Semantic Segmentation,比如SQ, SegNet, ENet等大大地提高了准确率，足以与Deeplab v2媲美，给语义分割的落地提供了可能。
+ICNet利用了低分辨率图片的高效处理和高分辨率图片的高推断质量两种优点。主要思想是：让低分辨率图像经过整个语义网络输出一个粗糙的预测，然后利用文中提出的级联融合单元来引入中分辨率和高分辨率图像的特征，从而逐渐提高精度。整个网络结构如下：


同上，和知乎的中文描述相同，需要修改。

qingqing01 · 2018-06-14T09:52:33Z

fluid/icnet/README.md

+
+
+
+本文采用Cityscape数据集，请前往[Cityscape官网]()注册下载。下载数据之后，按照[这里](https://github.com/mcordts/cityscapesScripts)的说明和工具处理数据。


进入https://github.com/mcordts/cityscapesScripts 这个链接里，一眼看去，看不懂如何处理数据，我们这里可以提供处理脚本吗？

qingqing01 · 2018-06-14T09:59:47Z

fluid/icnet/cityscape.py

+def test():
+    reader = DataGenerater(TEST_LIST).create_reader()
+    reader =  paddle.reader.map_readers(load, reader)
+    reader =  paddle.reader.map_readers(test_process, reader)


不需要两个 paddle.reader.map_readers吧。

qingqing01 · 2018-06-14T10:08:27Z

fluid/icnet/infer.py

+    # define network
+    images = fluid.layers.data(name='image', shape=data_shape, dtype='float32')
+    _, _, sub124_out = network.icnet(args, images, num_classes, np.array(data_shape[1:]).astype("float32"), is_test=True)
+    predict = fluid.layers.upsampling_bilinear2d(sub124_out, out_shape=data_shape[1:3])


upsampling_bilinear2d -> resize_bilinear

qingqing01 · 2018-06-14T10:10:17Z

fluid/icnet/infer.py

+    fluid.io.load_persistables(exe, args.model_path)
+    print "loaded model from: %s" % args.model_path
+    sys.stdout.flush()
+


因为网络中有BN，需要：

inference_program = fluid.default_main_program().clone(for_test=True)

下面83行的exe运行这个 inference_program

qingqing01 · 2018-06-14T10:11:29Z

fluid/icnet/network.py

+
+    sub4_out = conv(conv5_4_interp, 1, 1, num_classes, 1, 1, biased=True, relu=False, name="sub4_out")
+
+    sub24_out = conv(sub24_sum_interp, 1, 1, num_classes, 1, 1, biased=True, relu=False, name="sub24_out")


上面网络可以模块化的写下吗？现在看着网络特别复杂。

qingqing01 · 2018-06-14T10:11:50Z

fluid/icnet/network.py

@@ -0,0 +1,337 @@
+import paddle.fluid as fluid


network.py 有更精确的名字吗？

qingqing01 · 2018-06-14T10:17:01Z

https://travis-ci.org/PaddlePaddle/models/builds/390560755?utm_source=github_status&utm_medium=notification

另外，code style没过，需要安装 pre-commit : http://www.paddlepaddle.org/docs/develop/documentation/fluid/zh/dev/contribute_to_paddle_cn.html#permalink-5--pre-commit-

4luojing · 2018-06-18T15:16:19Z

已按review修改代码。请@wanghaoshuang @qingqing01 确认。

wanghaoshuang

LGTM. 不知道 @qingqing01 还有什么意见么？

Add icnet.

10d7980

qingqing01 changed the title ~~Add icnet.~~ Add ICNet. Jun 11, 2018

qingqing01 requested a review from wanghaoshuang June 11, 2018 06:03

qingqing01 changed the title ~~Add ICNet.~~ Add ICNet for image segmentation. Jun 11, 2018

wanghaoshuang requested changes Jun 12, 2018

View reviewed changes

wanghaoshuang reviewed Jun 14, 2018

View reviewed changes

qingqing01 reviewed Jun 14, 2018

View reviewed changes

Fix code.

ac03196

Fix utils.py.

fb04e92

wanghaoshuang approved these changes Jun 19, 2018

View reviewed changes

wanghaoshuang merged commit 2cb27d0 into PaddlePaddle:develop Jun 21, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add ICNet for image segmentation. #975

Add ICNet for image segmentation. #975

4luojing commented Jun 11, 2018

wanghaoshuang Jun 12, 2018

wanghaoshuang Jun 12, 2018

wanghaoshuang Jun 12, 2018

wanghaoshuang Jun 12, 2018

wanghaoshuang Jun 12, 2018

wanghaoshuang Jun 12, 2018

wanghaoshuang Jun 12, 2018

wanghaoshuang Jun 12, 2018

lgone2000 commented Jun 14, 2018

qingqing01 commented Jun 14, 2018

wanghaoshuang Jun 14, 2018

wanghaoshuang Jun 14, 2018

wanghaoshuang Jun 14, 2018

wanghaoshuang Jun 14, 2018

wanghaoshuang Jun 14, 2018

wanghaoshuang Jun 14, 2018

wanghaoshuang Jun 14, 2018

wanghaoshuang Jun 14, 2018

wanghaoshuang Jun 14, 2018

wanghaoshuang Jun 14, 2018

qingqing01 Jun 14, 2018

qingqing01 Jun 14, 2018

qingqing01 Jun 14, 2018

qingqing01 Jun 14, 2018

qingqing01 Jun 14, 2018 •

edited

qingqing01 Jun 14, 2018

qingqing01 Jun 14, 2018

qingqing01 Jun 14, 2018

qingqing01 Jun 14, 2018

qingqing01 Jun 14, 2018

qingqing01 commented Jun 14, 2018

4luojing commented Jun 18, 2018

wanghaoshuang left a comment •

edited

		return image_labels[0].astype("float32"), label_sub1, mask_sub1.astype("int32"), label_sub2, mask_sub2.astype("int32"), label_sub4, mask_sub4.astype("int32")


		def train(batch_size=32, random_mirror=False, random_scaling=False):

		import sys


		def conv(input, k_h, k_w, c_o, s_h, s_w, relu=True, padding="VALID", group=1, biased=None, name=None, print_pad=False):

		conv3_3_1_1_increase = conv(conv3_3_3_3_bn, 1, 1, 256, 1, 1, biased=False, relu=False, name="conv3_3_1_1_increase")
		conv3_3_1_1_increase_bn = bn(conv3_3_1_1_increase, relu=False, name="conv3_3_1_1_increase_bn", is_test=is_test)




		本文采用Cityscape数据集，请前往[Cityscape官网]()注册下载。下载数据之后，按照[这里](https://github.com/mcordts/cityscapesScripts)的说明和工具处理数据。


		## 简介

		Image Cascade Network（ICNet），在兼顾实时性的同时，比原来的Fast Semantic Segmentation,比如SQ, SegNet, ENet等大大地提高了准确率，足以与Deeplab v2媲美，给语义分割的落地提供了可能。


		sub4_out = conv(conv5_4_interp, 1, 1, num_classes, 1, 1, biased=True, relu=False, name="sub4_out")

		sub24_out = conv(sub24_sum_interp, 1, 1, num_classes, 1, 1, biased=True, relu=False, name="sub24_out")

Add ICNet for image segmentation. #975

Add ICNet for image segmentation. #975

Conversation

4luojing commented Jun 11, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lgone2000 commented Jun 14, 2018

qingqing01 commented Jun 14, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

qingqing01 Jun 14, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

qingqing01 commented Jun 14, 2018

4luojing commented Jun 18, 2018

wanghaoshuang left a comment • edited

Choose a reason for hiding this comment

qingqing01 Jun 14, 2018 •

edited

wanghaoshuang left a comment •

edited