Re-implement segnet in MXnet #8423

solin319 · 2017-10-25T08:01:53Z

Re-implement segnet in MXnet.
Details in https://github.com/solin319/incubator-mxnet/tree/solin-patch-segnet/example/segnet
The Camvid test accuracy achieved 83% in segnet_basic network.

Does MXNet need this work?

chinakook · 2017-10-26T00:53:12Z

Very nice work! At long last, Unpooling is reimplemented in MXNet. Segnet and U-Net are both famous in Kaggle and many practical applications rather than FCN-xs and Deeplab so I think MXNet need this work.

piiswrong · 2017-10-26T04:33:16Z

Does result match original paper?

@zhreshold Could you have a look?

zhreshold · 2017-10-27T20:44:47Z

Nice, can you put some instruction of how to use your operators? @solin319
And I cannot find the performance in either your repo or the original caffe one.

solin319 · 2017-10-28T01:10:46Z

Build MXNet with new pooling and upsampling operators.

# copy new operators to src/operator/
cp segnet/op/* incubator-mxnet/src/operator/
# rebuild MXNet from source
cd incubator-mxnet/
make
cd python/
python setup.py install

zhreshold · 2017-11-01T18:11:41Z

example/segnet/common/contrib_metrics.py

+
+            mx.metric.check_label_shapes(label, pred_label)
+
+            self.sum_metric += (pred_label.flat == label.flat).sum()


If ignore_label is in pred_label, e.g. ignore_label == 0, this is wrong.

elf.sum_metric -= len(pred_label[pred_label == self.ignore_label])

This can solve the problem caused by ignore_label located in pred_label.

zhreshold · 2017-11-01T18:17:38Z

example/segnet/common/contrib_metrics.py

+            output_names=output_names, label_names=label_names)
+        self.eps = eps
+
+    def update(self, labels, preds):


Why not use the native one in metric.py?

1.The native 'CorssEntropy' in metric.py can only used to pred_label with 2-d.
2. In this example, the pred_label is 4-d and the softmax result located in axis 1. We must swap the softmax result to the last axis.
3. Because of the ignore_label=11, the max number of label is 11, but the size of softmax result is only 0-10. We must add one column data to use the command
prob = pred[numpy.arange(label.shape[0]), numpy.int64(label)]

zhreshold · 2017-11-01T18:18:24Z

example/segnet/data_iter.py

+        return list(data.items()), list(label.items())
+
+    def _read_img(self, img_name, label_name):
+        img = Image.open(os.path.join(self.root_dir, img_name))


You can use mx.image.imread instead of PIL package.

zhreshold · 2017-11-01T18:20:58Z

example/segnet/op/pooling_mask_max_2d.cc

+
+DMLC_REGISTER_PARAMETER(PoolingMaskParam);
+
+MXNET_REGISTER_OP_PROPERTY(PoolingMask, PoolingMaskProp)


Do you think this operator is reusable to be put into operator/contrib?

zhreshold · 2017-11-01T18:21:19Z

example/segnet/op/upsampling_mask_max_2d.cc

+
+DMLC_REGISTER_PARAMETER(UpSamplingMaskParam);
+
+MXNET_REGISTER_OP_PROPERTY(UpSamplingMask, UpSamplingMaskProp)


Do you think this operator is reusable to be put into operator/contrib?

zhreshold · 2017-11-01T18:22:13Z

example/segnet/score.py

+import logging
+import pdb
+import numpy as np
+from PIL import Image


Same here, use mx.image

zhreshold · 2017-11-01T18:22:40Z

example/segnet/segment_one_pic.py

+import os
+import logging
+import numpy as np
+from PIL import Image


zhreshold

See comments.
Remove result pngs, you can use github issue to upload images and get permanent links.

solin319 · 2017-11-02T07:05:52Z

@zhreshold
Thank you for your suggestions. I have use mx.image to read pic and remove results pngs. But I still have two questions.

I think the operator pooling and up-sampling with mask may used in other pixelwise segmentation networks. Shall I move them to operator/contrib.
I use PIL to save numpy array to colorful png. I can't find the interface in mx.image.

zhreshold · 2017-11-02T18:38:40Z

@solin319
Yes, move to operator/contrib if they are reusable.
Do you really need to save pics? If so, it's fine to leave it there.
And BTW, do you think your result can match the paper?

solin319 · 2017-11-03T08:49:41Z

@zhreshold
Yes, the test result can match the paper.
In CamVid data-set, we used 367 for training and 233 for testing like paper.
The train command is python train_segnet.py --gpus 0,1,2,3 --lr=0.01 --network=segnet_basic .
num_epochs = 250
lr_steps = 5000,7000
The result was shown below.

We also trained with segnet_basic_with_drop, the result was shown below.
num_epochs = 400
lr_steps = 9000,10000

The test accuracy was similar to the paper.

zhreshold · 2017-11-08T01:05:00Z

@solin319 Have you finished? Can you rebase to fix the CI?

piiswrong · 2017-11-11T20:12:56Z

Result on the official site is:

Model	Global Accuracy	Class Accuracy	Mean I/U
Segnet-Basic	82.8%	62.3%	46.3%
SegNet (Pretrained Encoder)	88.6%	65.9%	50.2%
SegNet (3.5K dataset)	86.8%	81.3%	69.1%

Which one are you comparing to?

solin319 · 2017-11-12T09:31:43Z

The first one segnet-basic, training with 367 pics and test in 233 pics.
@piiswrong

piiswrong · 2017-12-12T22:19:03Z

@zhreshold @winstywang @solin319 Any updates on this?

solin319 · 2017-12-26T07:56:34Z

We trained segnet with vgg16-bn pretrained parameters and got accuracy 0.89.

@piiswrong

piiswrong · 2018-01-31T19:59:44Z

@zhreshold Could you check if this reproduces the paper?

zhreshold · 2018-02-01T03:36:03Z

@solin319 Is this table SOTA as you are results are comparing with?

solin319 · 2018-02-01T06:07:09Z

comparing with:

Model	Global Accuracy	Class Accuracy	Mean I/U
Segnet-Basic	82.8%	62.3%	46.3%
SegNet (Pretrained Encoder)	88.6%	65.9%	50.2%

http://mi.eng.cam.ac.uk/projects/segnet/tutorial.html
@zhreshold

CodingCat · 2018-03-07T04:46:01Z

Hi, the community has passed to vote about associating the code changes with JIRA (https://lists.apache.org/thread.html/ab22cf0e35f1bce2c3bf3bec2bc5b85a9583a3fe7fd56ba1bbade55f@%3Cdev.mxnet.apache.org%3E)

We have updated the guidelines for contributors in https://cwiki.apache.org/confluence/display/MXNET/Development+Process, please ensure that you have created a JIRA at https://issues.apache.org/jira/projects/MXNET/issues/ to describe your work in this pull request and include the JIRA title in your PR as [MXNET-xxxx] your title where MXNET-xxxx is the JIRA id

Thanks!

solin319 changed the title ~~Solin patch segnet~~ Re-implement segnet in MXnet Oct 25, 2017

zhreshold reviewed Nov 1, 2017

View reviewed changes

example/segnet/score.py

import logging

import pdb

import numpy as np

from PIL import Image

Copy link

Member

zhreshold Nov 1, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, use mx.image

zhreshold reviewed Nov 1, 2017

View reviewed changes

example/segnet/segment_one_pic.py

import os

import logging

import numpy as np

from PIL import Image

Copy link

Member

zhreshold Nov 1, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here.

zhreshold reviewed Nov 1, 2017

View reviewed changes

solin319 and others added 12 commits November 8, 2017 20:22

segnet symbol

5d5f118

segnet framework

ec72adc

pool with mask

5f04838

res pic

4e6965f

Update README.md

6c6015d

Add files via upload

5e70d1e

Update README.md

c1637c9

add license

464a88f

use mx.imread remove result pic

7533818

acc metric

8b1193e

update train_segnet

9986338

add segnet_basic_with_dropout

0bee468

solin319 force-pushed the solin-patch-segnet branch from 2c42c9a to 0bee468 Compare November 8, 2017 12:23

solin319 added 2 commits December 26, 2017 15:52

Update segnet.py

e80035c

Update README.md

6a4172a

solin319 added 2 commits December 27, 2017 09:27

Add bayesian segnet

e712b2e

Add Bayesian test result in readme

40e68f5

solin319 closed this Apr 27, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Re-implement segnet in MXnet #8423

Re-implement segnet in MXnet #8423

solin319 commented Oct 25, 2017

chinakook commented Oct 26, 2017 •

edited

Loading

piiswrong commented Oct 26, 2017

zhreshold commented Oct 27, 2017

solin319 commented Oct 28, 2017 •

edited

Loading

zhreshold Nov 1, 2017

solin319 Nov 2, 2017

zhreshold Nov 1, 2017

solin319 Nov 2, 2017

zhreshold Nov 1, 2017

zhreshold Nov 1, 2017

zhreshold Nov 1, 2017

zhreshold Nov 1, 2017

zhreshold Nov 1, 2017

zhreshold left a comment

solin319 commented Nov 2, 2017

zhreshold commented Nov 2, 2017

solin319 commented Nov 3, 2017 •

edited

Loading

zhreshold commented Nov 8, 2017

piiswrong commented Nov 11, 2017

solin319 commented Nov 12, 2017

piiswrong commented Dec 12, 2017

solin319 commented Dec 26, 2017 •

edited

Loading

piiswrong commented Jan 31, 2018

zhreshold commented Feb 1, 2018

solin319 commented Feb 1, 2018 •

edited

Loading

CodingCat commented Mar 7, 2018


		mx.metric.check_label_shapes(label, pred_label)

		self.sum_metric += (pred_label.flat == label.flat).sum()


		DMLC_REGISTER_PARAMETER(PoolingMaskParam);

		MXNET_REGISTER_OP_PROPERTY(PoolingMask, PoolingMaskProp)


		DMLC_REGISTER_PARAMETER(UpSamplingMaskParam);

		MXNET_REGISTER_OP_PROPERTY(UpSamplingMask, UpSamplingMaskProp)

Re-implement segnet in MXnet #8423

Re-implement segnet in MXnet #8423

Conversation

solin319 commented Oct 25, 2017

chinakook commented Oct 26, 2017 • edited Loading

piiswrong commented Oct 26, 2017

zhreshold commented Oct 27, 2017

solin319 commented Oct 28, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zhreshold left a comment

Choose a reason for hiding this comment

solin319 commented Nov 2, 2017

zhreshold commented Nov 2, 2017

solin319 commented Nov 3, 2017 • edited Loading

zhreshold commented Nov 8, 2017

piiswrong commented Nov 11, 2017

solin319 commented Nov 12, 2017

piiswrong commented Dec 12, 2017

solin319 commented Dec 26, 2017 • edited Loading

piiswrong commented Jan 31, 2018

zhreshold commented Feb 1, 2018

solin319 commented Feb 1, 2018 • edited Loading

CodingCat commented Mar 7, 2018

chinakook commented Oct 26, 2017 •

edited

Loading

solin319 commented Oct 28, 2017 •

edited

Loading

solin319 commented Nov 3, 2017 •

edited

Loading

solin319 commented Dec 26, 2017 •

edited

Loading

solin319 commented Feb 1, 2018 •

edited

Loading