Towards Robust Few-shot Point Cloud Semantic Segmentation

Created by Yating Xu from National University of Singapore

Introduction

This repository contains the PyTorch implementation for our BMVC 2023 Paper "Towards Robust Few-shot Point Cloud Semantic Segmentation" by Yating Xu, Conghui Hu, Na Zhao, Gim Hee Lee.

Installation

• python 3.8
• pytorch 1.8 with CUDA 11.1
• tensorboard, h5py, transforms3d

Usage

Data preparation

S3DIS

We follow attMPTI to process clean data. Data folder for S3DIS is like follow:

-- S3DIS
    |-- scenes
        |-- data
    |-- meta
    |--blocks_bs1.0_s1.0
        |-- data (block data)
        |--class2scans.pkl (Will be generated by running mpti_train_noise.py the first time.)
        |-- S_0_N_2_K_5_episodes_100_pts_2048 (validation data. Will be generated by running mpti_train_noise.py the first time.)
        |-- NoiseTest_ood_0.400000_S_0_N_2_K_5_test_episodes_100_pts_2048 (test data. Will be generated by eval_noise.py)
        ...

ScanNet

Similar to processing S3DIS.

Running

Training

1. Pre-training:
   We follow attMPTI to pre-train feature extractor on the clean meta-train dataset.

2. Training on S3DIS 2-way 5-shot:

python mpti_train_noise.py --use_attention --pc_augm --cvfold 0  
--dataset s3dis --clean_data_path datasets/S3DIS/blocks_bs1_s1    
--pretrain_checkpoint_path attMPTI-clean/log_s3dis/log_pretrain_s3dis_S0/checkpoint.tar  
--n_way 2 --k_shot 5 --train_noise_ratio [0,0.2,0.4] 
--log_dir mpti_WayContrast=0.1+FPS=4_V2_[0,0.2,0.4]   --save_path log_s3dis

Evaluation

Evaluation on S3DIS 2-way 5-shot:

python eval_noise.py --dataset s3dis --clean_data_path datasets/S3DIS/blocks_bs1_s1
--model_checkpoint_path log_s3dis/Cleantrain/S0_N2_K5_Att1/mpti_WayContrast=0.1+FPS=4_V2_[0,0.2,0.4]/  --use_attention 
--n_way 2 --k_shot 5 --phase mptinoise_eval --cvfold 0 
--noise_ratio 0.4 --noise_type ood

Change noise_ratio and noise_type for evaluation on other noise settings. Set noise_ratio=0 for evaluation on clean meta-test dataset.

Acknowledgement

We thank attMPTI for sharing their source code.