👉 News: Last update: 02/06, approximate to finish it before 02/21/2023

Towards-OpenVAD

Yuansheng Zhu, Wentao Bao, and Qi Yu

This is the official implementation of our paper Towards Open Set Video Anomaly Detection in ECCV 2022.

Abstract: Open Set Video Anomaly Detection (OpenVAD) aims to identify abnormal events from video data where both known anomalies and novel ones exist in testing. Unsupervised models learned solely from normal videos are applicable to any testing anomalies but suffer from a high false positive rate. In contrast, weakly supervised methods are effective in detecting known anomalies but could fail in an open world. We develop a novel weakly supervised method for the OpenVAD problem by integrating evidential deep learning (EDL) and normalizing flows (NFs) into a multiple instance learning (MIL) framework. Specifically, we propose to use graph neural networks and triplet loss to learn discriminative features for training the EDL classifier, where the EDL is capable of identifying the unknown anomalies by quantifying the uncertainty. Moreover, we develop an uncertainty-aware selection strategy to obtain clean anomaly instances and a NFs module to generate the pseudo anomalies. Our method is superior to existing approaches by inheriting the advantages of both the unsupervised NFs and the weakly-supervised MIL framework. Experimental results on multiple real-world video datasets show the effectiveness of our method.

Experiments

Data preparation

• The raw data of three datasets could be found in: Xd-Violence, UCF-Crime, and ShanghaiTech Campus.
• The extracted features of UCF and ShanghaiTech is from link, and Xd-Violence is from link
• The details of split could be found in Dropbox. To create the OpenVAD scenario which mimics the natural characteristics of anomalous events, we resplit the data to create novel anomaly in test data.

Running environment

Creat enviroment

Running

Make list and gt

Citation

@inproceedings{zhu2022towards,
  title={Towards Open Set Video Anomaly Detection},
  author={Zhu, Yuansheng and Bao, Wentao and Yu, Qi},
  booktitle={Computer Vision--ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23--27, 2022, Proceedings, Part XXXIV},
  pages={395--412},
  year={2022},
  organization={Springer}
}

Acknowledgements

This project contains codes from following repostories:

• Xd-Violence
• RTFM
• Posterior Network.

We sincirelly thank for their great efforts!