This is not the official repository for PIDNet (PDF)
- Without remove logging, we added tqdm for better visualization of training, valisation and evaluation
- Added output of precision, recall and F1-score
- Improved readability and versatility of parameters
- Improved file path handling
- Improved loading of dataset (support single image, video, and directory), added visualize and show results during inference
Comparison of inference speed and accuracy for real-time models on test set of Cityscapes.
- Towards Real-time Applications: PIDNet could be directly used for the real-time applications, such as autonomous vehicle and medical imaging.
- A Novel Three-branch Network: Addtional boundary branch is introduced to two-branch network to mimic the PID controller architecture and remedy the overshoot issue of previous models.
- More Accurate and Faster: PIDNet-S presents 78.6% mIOU with speed of 93.2 FPS on Cityscapes test set and 80.1% mIOU with speed of 153.7 FPS on CamVid test set. Also, PIDNet-L becomes the most accurate one (80.6% mIOU) among all the real-time networks for Cityscapes.
A demo of the segmentation performance of PIDNets with our dataset: Original video (left) and predictions of PIDNet-S (right)
Oil Pollution Sementic Segmentation demo video
An overview of the basic architecture of Proportional-Integral-Derivative Network (PIDNet).
For simple reproduction, here provided the ImageNet pretrained models. Also, the finetuned models on Oil Pollution are available for direct application in marine oil pollution detection.
| Model | Links |
|---|---|
| ImageNet Pretrained | PIDNet-S |
| Finetuned Oil Pollution | PIDNet-S |
| Data Augmentation | Mean IoU (%) | Pixel Accuracy (%) | Mean Accuracy (%) |
|---|---|---|---|
| NA | 78.71 | 94.94 | 86.80 |
| HSV | 85.71 | 96.54 | 93.30 |
| HSV & MBA | 86.63 | 96.72 | 93.69 |
| Data Augmentation | Class IoUs (%) | Sky | Ship | Ocean | Wave | Shore | Oil | Mean | |
|---|---|
| NA | | 94.85 | 89.80 | 93.88 | 57.42 | 73.99 | 62.34 | 78.71 | |
| HSV | | 96.22 | 94.33 | 95.64 | 64.14 | 89.54 | 74.40 | 85.71 | |
| HSV & MBA | | 96.44 | 94.45 | 95.84 | 65.71 | 91.60 | 75.72 | 86.63 | |
| Data Augmentation | F1-Scores | Sky | Ship | Ocean | Wave | Shore | Oil | Mean | |
|---|---|
| NA | | 0.973 | 0.946 | 0.968 | 0.729 | 0.850 | 0.768 | 0.873 | |
| HSV | | 0.981 | 0.971 | 0.978 | 0.782 | 0.945 | 0.853 | 0.918 | |
| HSV & MBA | | 0.982 | 0.971 | 0.979 | 0.793 | 0.956 | 0.862 | 0.923 | |
| Oil Pollution Dataset | Links |
|---|---|
| Dataset | Download |
| Config | Download |
| ClassName | Label |
|---|---|
| sky | 19 |
| ship | 20 |
| ocean | 21 |
| wave | 22 |
| shore | 23 |
| oil | 24 |
- Converts the hue value of the oil part by gamma correction so that it can be adjusted to other colors
Gamma Correction as Data Augmentation
- By presenting the distribution of HSV values to compare whether effective data augmentation can be achieved by modifying the Hue values
As you can see, our training data set has the most concentrated distribution of hues, so we decided to augment the data for the best results by targeting hues
- This data augmentation method was proposed by LIGHT-WEIGHT MIXED STAGE PARTIAL NETWORK FOR SURVEILLANCE OBJECT DETECTION WITH BACKGROUND DATA AUGMENTATION (pdf)
- To include foreground information obtained by background subtraction to generate more training samples so that the learner can learn important features only around foreground objects.
Mixing Background and original image from α = 10% to 30%
- Clone this repository or offical repository
- Download the OilPollution datasets and configuration files, unzip them and replace in
rootdir. - Check if the paths contained in lists of
data/listare correct for dataset images.
- Download the ImageNet pretrained models and put them into
pretrained_models/imagenet/dir. - For example, train the PIDNet-S on OilPollution with on 2 GPUs:
python3 tools/train.py --cfg configs/oilpollution/pidnet_small_oil_HSV_MBA.yaml GPUS "(0,1)"- Evaluate pretrained PIDNet-S with oil pollution dataset
python3 tools/eval.py --cfg configs/oilpollution/pidnet_small_oil_HSV_MBA.yaml TEST.MODEL_FILE output/oilpollution/pidnet_small_oil_HSV_MBA/best.pt
- check offical repository
- Put all your images in
samples/and then run the command below using OilPollution pretrained PIDNet-S for all files insamples/:
python3 tools/custom.py --model-type 'pidnet-s' --model output/oilpollution/pidnet_small_oil_HSV_MBA/best.pt --input samples --n-class 6- Using OilPollution pretrained PIDNet-L for all files in
samples/and show visualized results:
python3 tools/custom.py --model-type 'pidnet-l' --model output/oilpollution/pidnet_small_oil_HSV_MBA/best.pt --input samples --n-class 6 --visualize --show- Our implementation is based on PIDNet: A Real-time Semantic Segmentation Network Inspired from PID Controller.
- Thanks for their nice contribution.