Skip to content

CVPR2022 - Deep Hierarchical Semantic Segmentation - A structured, pixel-wise description of visual scenes in terms of the class hierarchy.

Notifications You must be signed in to change notification settings

lingorX/HieraSeg

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

9 Commits
 
 
 
 
 
 
 
 

Repository files navigation

HieraSeg (CVPR'22)

Liulei Li, Tianfei Zhou, Wenguan Wang†, Jianwu Li, Yi Yang

[arXiv] [BibTeX]

Updates

  • The Pytorch implementation has already been released!
  • New work GMMSeg, the first generative neural framework for semantic segmentation has been accepted by NeurIPS2022! [code for GMMSeg]
  • This repo will release an official PaddlePaddle implementation for paper: Deep Hierarchical Semantic Segmentation.

Abstract

Humans are able to recognize structured relations in observation, allowing us to decompose complex scenes into simpler parts and abstract the visual world in multiple levels. However, such hierarchical reasoning ability of human perception remains largely unexplored in current literature of semantic segmentation. Existing work is often aware of flatten labels and predicts target classes exclusively for each pixel. In this paper, we instead address hierarchical semantic segmentation (HSS), which aims at structured, pixel-wise description of visual observation in terms of a class hierarchy. We devise HSSN, a general HSS framework that tackles two critical issues in this task: i) how to efficiently adapt existing hierarchy-agnostic segmentation networks to the HSS setting, and ii) how to leverage the hierarchy information to regularize HSS network learning. To address i), HSSN directly casts HSS as a pixel-wise multi-label classification task, only bringing minimal architecture change to current segmentation models. To solve ii), HSSN first explores inherent properties of the hierarchy as a training objective, which enforces segmentation predictions to obey the hierarchy structure. Further, with hierarchy-induced margin constraints, HSSN reshapes the pixel embedding space, so as to generate well-structured pixel representations and improve segmentation eventually. We conduct experiments on four semantic segmentation datasets (i.e., Mapillary Vistas 2.0, Cityscapes, LIP, and PASCAL-Person-Part), with different class hierarchies, segmentation network architectures and backbones, showing the generalization and superiority of HSSN.

Other Related Work

GMMSeg: Gaussian Mixture based Generative Semantic Segmentation Models

Exploring Cross-Image Pixel Contrast for Semantic Segmentation (ICCV21-Oral)

Rethinking Semantic Segmentation: A Prototype View (CVPR22-Oral)

Citing HieraSeg

@article{li2022deep,
  title={Deep Hierarchical Semantic Segmentation},
  author={Li, Liulei and Zhou, Tianfei and Wang, Wenguan and Li, Jianwu and Yang, Yi},
  journal={arXiv preprint arXiv:2203.14335},
  year={2022}
}

About

CVPR2022 - Deep Hierarchical Semantic Segmentation - A structured, pixel-wise description of visual scenes in terms of the class hierarchy.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages