ImportError: No module named 'inplace_abn'

[InternetMaster1]

I tried Image Segmentation using the " HRNetV2-W18-Small-v2 " small model with cityscape dataset.

I haveinstall all modules mentioned in requirement.txt file with the matching version of the modules.
My config is as follows -
python 3.6
cuda 9.2
ninja 1.8.2
pytorch 0.4.1

I had done the steps till data preparations and then I tried to train using following command,

python tools/train.py --cfg experiments/cityscapes/seg_hrnet_w18_small_v1_512x1024_sgd_lr1e-2_wd5e-4_bs_12_epoch484

I am getting the below error.

warnings.warn(ABI_INCOMPATIBILITY_WARNING.format(compiler)) Traceback (most recent call last): 
File "/usr/local/lib/python3.6/dist-packages/torch/utils/cpp_extension.py", line 759, in _build_extension_module ['ninja', '-v'], stderr=subprocess.STDOUT, cwd=build_directory) 
File "/usr/lib/python3.6/subprocess.py", line 356, in check_output **kwargs).stdout 
File "/usr/lib/python3.6/subprocess.py", line 438, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/torch/utils/cpp_extension.py", line 514, in load with_cuda=with_cuda) 
File "/usr/local/lib/python3.6/dist-packages/torch/utils/cpp_extension.py", line 690, in _jit_compile return _import_module_from_library(name, build_directory) 
File "/usr/local/lib/python3.6/dist-packages/torch/utils/cpp_extension.py", line 770, in _import_module_from_library file, path, description = imp.find_module(module_name, [path]) 
File "/usr/lib/python3.6/imp.py", line 297, in find_module raise ImportError(_ERR_MSG.format(name), name=name)
ImportError: No module named 'inplace_abn'

To solve this, I have tried with diff version that match with ninja and cuda, but no luck. Any help please!

[InternetMaster1]

Any help please...

@pgrimaud @leoxiaobin @sunke123 @bearcatt

[sunke123]

The HRNet small model is trained based on Pytorch-v1.1 and official sync-bn is used (inplace-abn is not used here).

You can try our pytorch-v1.1 branch.

[InternetMaster1]

@sunke123 Many thanks, I will give it a try!

Just a couple more questions :

1. Is there "HRNet + OCR + SegFix" option available for HRNetV2-W18-Small-v2?

The addition of OCR & Segfix would improve the accuracy of "HRNetV2-W18-Small-v2" further?

2. Is "HRNetV2-W18-Small-v2" SOTA for lightweight segmentation?

Comparison of mIOU on Cityscapes.

• HRNetV2-W18-Small-v2 : 76.2%
• U-HarDNet-70 : 75.9%

Why is U-HarDNet-70 at the top of paperswithcode SOTA list for real-time segmentation?

I am looking to implement high-quality semantic segmentation on a mobile device for accurate human segmentation for still images (i.e. non-realtime). Would "HRNetV2-W18-Small-v2" be a great option for this?

[InternetMaster1]

@sunke123

I tried pytorch-v1.1 branch and installed the following exact configuration 👍

PyTorch=1.1.0
EasyDict==1.7
opencv-python==3.4.1.15
shapely==1.6.4
Cython
scipy
pandas
pyyaml
json_tricks
scikit-image
yacs>=0.1.5
tensorboardX>=1.6
tqdm
ninja
given in following link
https://github.com/HRNet/HRNet-Semantic-Segmentation/blob/pytorch-v1.1/requirements.txt

I am still getting same error "No module named 'inplace_abn'" while training

You said

The HRNet small model is trained based on Pytorch-v1.1 and official sync-bn is used (inplace-abn is not used here)

but in this file, I noticed that "inplace_abn" is imported

from .inplace_abn import bn

Do I need to do any changes here?

[sunke123]

@InternetMaster1
You can remove the sync-bn folder directly.

1. Is there "HRNet + OCR + SegFix" option available for HRNetV2-W18-Small-v2?
   I have not tried this, but I think is is OK.
   @hsfzxjy Could you try to implement OCR module on the HRNetV2-W18-Small-v2?

2. Is "HRNetV2-W18-Small-v2" SOTA for lightweight segmentation?
   In terms of performance, HRNetV2-W18-Small-v2 is actually the best model, for now.
   But HRNetV2-W18-Small-v2 is not designed for lightweight segmentation, maybe speed is lower than U-HarDNet-70.
   My teammate has a new work based on HRNet for lightweight segmentation, and will release the code recently. You can check that.

3. I am looking to implement high-quality semantic segmentation on a mobile device for accurate human segmentation for still images (i.e. non-realtime). Would "HRNetV2-W18-Small-v2" be a great option for this?

I think that HRNet is a good choice.
In my opinion, HRNet achieves good performance on segmentation and human pose estimation, accurate prediction of keypoint should also help the human seg.

[hsfzxjy]

@InternetMaster1 I think you can implement HRNetV2-W18-Small-v2 + OCR by overriding MODEL.EXTRA in https://github.com/HRNet/HRNet-Semantic-Segmentation/blob/HRNet-OCR/experiments/cityscapes/seg_hrnet_ocr_w48_train_512x1024_sgd_lr1e-2_wd5e-4_bs_12_epoch484.yaml with the config of HRNetV2-W18-Small-v2.

You may also need to override MODEL.OCR.MID_CHANNELS and MODEL.OCR.KEY_CHANNELS. These two config items are not explicitly declared in .yaml file, but you can find them here. Practically, MODEL.OCR.MID_CHANNELS should be a bit lower than the final output channels of HRNet (default to 512 for HRNet-W48 with 720 channels), and KEY_CHANNELS be half of MID_CHANNELS. I suggest to set MODEL.OCR.MID_CHANNELS = 256 and MODEL.OCR.KEY_CHANNELS = 128.

SegFix is another standalone post-processing mechenism. You can find its usage in our openseg repository.

[InternetMaster1]

@sunke123

Thank you for the plethora of information! Yes, HRNet looks like an awesome option for our needs!

1. You can remove the sync-bn folder directly.

Can you be more specific? Along with deletion of the folder, do I need to even make any changes in any py files? Sorry for the newbie question, but I am stuck at installation

2. When is the new work based on HRNet for lightweight segmentation expected to release?

[InternetMaster1]

@hsfzxjy

Many thanks for the detailed informtion. Once my installation is successful, I will check out the OCR option!

1. If you get a chance, could you add a yaml file for the same? Could be useful to other users too. Thanks.

2. Do you think using the OCR option will further improve the performance of HRNetV2-W18-Small-v2 and give a further better mIOU than the current mentioned 76.2?

3. What is the main purpose for SegFix? It is for smoothing of the edges?

[sunke123]

@InternetMaster1

InplaceABN is not used in the code of the Pytorch v-1.1 branch.
So, you can remove this folder without any changes.

The lightweight segmentation work is submitted to a CV conference.
I think that they will release the code after the acceptance.

By the way, the author of lightweight hrnet tell me that you can try our HRNetV2-W18-Small-v2. It's actually a good choice.

[InternetMaster1]

@sunke123

I was able to finally get it!

I was trying to use clone method from the pytorch-v1.1 branch. The URL for clone of the branch is same as the main repo, and hence the confusion!
https://github.com/HRNet/HRNet-Semantic-Segmentation.git

I had to use the Download functionality to get the code for the branch
https://github.com/HRNet/HRNet-Semantic-Segmentation/archive/pytorch-v1.1.zip

Thanks

[InternetMaster1]

@hsfzxjy

If you get a chance, could you reply to the following, many thanks in advance

1. Do you think using the OCR option will further improve the performance of HRNetV2-W18-Small-v2 and give a further better mIOU than the current mentioned 76.2?
2. What is the main purpose for SegFix? It is for smoothing of the edges?
3. If you get a chance, could you add a yaml file for HRNetV2-W18-Small-v2 + OCR combination? Could be useful to other users too. Thanks.

[InternetMaster1]

@sunke123

How to convert the final model file to make it work on mobile, mainly Android?(i.e. tensorflow lite or ncnn). Are there any special steps involved?

I don't have any experience with HRNet and your guidance would be most helpful

Many thanks!

[ywang370]

@InternetMaster1

InplaceABN is not used in the code of the Pytorch v-1.1 branch.
So, you can remove this folder without any changes.

The lightweight segmentation work is submitted to a CV conference.
I think that they will release the code after the acceptance.

By the way, the author of lightweight hrnet tell me that you can try our HRNetV2-W18-Small-v2. It's actually a good choice.

Hi I found 1.1 branch actually use batchnorm2d, I guess it is not syncnorm. Dose it mean no difference?

[hsfzxjy]

@InternetMaster1
InplaceABN is not used in the code of the Pytorch v-1.1 branch.
So, you can remove this folder without any changes.
The lightweight segmentation work is submitted to a CV conference.
I think that they will release the code after the acceptance.
By the way, the author of lightweight hrnet tell me that you can try our HRNetV2-W18-Small-v2. It's actually a good choice.

Hi I found 1.1 branch actually use batchnorm2d, I guess it is not syncnorm. Dose it mean no difference?

@ywang370 No. We use nn.SyncBatchNorm for PyTorch 1.1. See here.

[sunke123]

@InternetMaster1

I also have no experience with the mobile application.
Sorry for that

[InternetMaster1]

Thank you @sunke123 & @hsfzxjy

You guys rock!