Skip to content
main
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Learnable Tree Filter V2

By Lin Song, Yanwei Li, Zhengkai Jiang, Zeming Li, Xiangyu Zhang, Hongbin Sun, Jian Sun, Nanning Zheng.

This repo is an official implementation for "Rethinking Learnable Tree Filter for Generic Feature Transform" (NeurIPS2020) on PyTorch framework.

introduce image

Installation

Requirements

  • Python >= 3.6
  • PyTorch >= 1.6 and torchvision
  • OpenCV, needed by demo and visualization
  • pycocotools:
    • pip install cython
    • pip install 'git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI'
  • prodict:
    • pip install prodict
  • GCC >= 4.9

Build from source

  • git clone https://github.com/StevenGrove/LearnableTreeFilterV2
  • cd LearnableTreeFilterV2
  • python setup.py build develop

Prepare data

See datasets/README.md.

Usage

All the projects are placed in cvpods_playground. You need to select a project and enter the corresponding folder.

# For example
cd cvpods_playground/mask_rcnn.res50.fpn.coco.1x.nn_syncbn.tf-v2

Training

# Running training procedure with specific GPU number
cvpods_train --gpu-nums <GPU_NUM> [optional arguments]

# Please refer to tools/train_net.py for more optional arguments

Inference

# Running inference procedure with specific GPU number
# The program will load the lastest model weights from ./log folder
cvpods_test --gpu-nums <GPU_NUM> [optional arguments]

# Running inference procedure with specific GPU number and model path
cvpods_test --gpu-nums <GPU_NUM> MODEL.WEIGHTS <model_path> [optional arguments]

# Please refer to tools/test_net.py for more optional arguments

Visualize your image [Optional]

# Please refer to demo/demo.py for more optional arguments
python ../../demo/demo.py --config <config_path> --input <input_path> --output <output_path> MODEL.WEIGHTS <model_path> [optional arguments]

Add to your code [Optional]

# Load the learnable tree filter module
from cvpods.layers import TreeFilterV2

# Initialize the module with specific number of channels and groups
tf = TreeFilterV2(<guide_channels>, <in_channels>, <embed_channels>, [num_groups])

# Run the filter procedure with input feature and guided feature
output_feature = tf(<input_feature>, <guided_feature>)

Performance

Instance Segmentation on COCO val set

Method Backbone Schedule APbox (%) APmask (%) Project Model
Mask-RCNN + LTF-V2 ResNet-50 1x 41.3 37.0 Link OneDrive
Mask-RCNN + LTF-V2 ResNet-101 1x 42.5 38.0 Link OneDrive
Mask-RCNN + LTF-V2 ResNeXt-101 1x 44.5 39.5 Link OneDrive
Cascade Mask-RCNN + LTF-V2 ResNeXt-101 1x 46.9 40.3 Link OneDrive

Panoptic Segmentation on COCO val set

Method Backbone Schedule PQ (%) APbox (%) APmask (%) Project Model
Panoptic-FPN ResNet-50 1x 39.2 37.4 33.6 Link OneDrive
Panoptic-FPN + LTF-V2 ResNet-50 1x 41.7 39.3 35.4 Link OneDrive

More Results is Coming

To do

  • Experiments on panoptic segmentation
  • Easier installation based on pip
  • Modify transformer
  • Application on video tasks

Citation

Please cite the paper in your publications if it helps your research.

@inproceedings{song2020rethinking,
    title = {Rethinking Learnable Tree Filter for Generic Feature Transform},
    author = {Song, Lin and Li, Yanwei and Jiang, Zhengkai and Li, Zeming and Zhang, Xiangyu and Sun, Hongbin and Sun, Jian and Zheng, Nanning},
    booktitle = {Advances in Neural Information Processing Systems},
    year = {2020}
}

Please cite this project in your publications if it helps your research.

@misc{learnabletreefilterv2,
    author = {Song, Lin},
    title = {LearnableTreeFilterV2},
    howpublished = {\url{https://github.com/StevenGrove/LearnableTreeFilterV2}},
    year = {2020}
}

About

No description, website, or topics provided.

Resources

License

Releases

No releases published

Packages

No packages published