Oral Presentation
The repository discusses the implementation of the paper
"CLIP-TSA: CLIP-Assisted Temporal Self-Attention for Weakly-Supervised Video Anomaly Detection"
using the PyTorch framework.
"CLIP-TSA: CLIP-Assisted Temporal Self-Attention for Weakly-Supervised Video Anomaly Detection"
using the PyTorch framework.
CLIP-TSA: CLIP-Assisted Temporal Self-Attention for Weakly-Supervised Video Anomaly Detection (Oral, ICIP 2023)
Kevin Hyekang Joo, Khoa Vo, Kashu Yamazaki, Ngan Le
- pytorch
- matplotlib
- tqdm
- scipy
- scikit-learn
Q1) I get the following error: "RuntimeError: Expected a 'cuda' device type for generator but found 'cpu'"
- A1) Please go to venv/lib/python3.8/site-packages/torch/utils/data/sampler.py, and find __iter__ function within RandomSampler class. Then change the line
generator = torch.Generator()togenerator = torch.Generator(device="cuda").
Q2) I keep getting CUDA OUT OF MEMORY error
- A2) Each dataset requires varying amounts of VRAM, and a significant amount of VRAM is expected to be used with the TSA feature enabled. Thus, please be advised if you want to run tests on big public datasets such as ShanghaiTech Campus, XD-Violence, and UCF-Crime Datasets. If you would like to test out only the power of CLIP within the model, please disable the TSA by adding
--disable_HAto the command, which requires less amount of VRAM and should be operable on most GPUs.
python main.py
Please change the hyperparameters & parameters accordingly by first looking at the main.py file. Otherwise, it will be run under default settings.
@inproceedings{joo2023cliptsa,
title={CLIP-TSA: CLIP-Assisted Temporal Self-Attention for Weakly-Supervised Video Anomaly Detection},
author={Joo, Hyekang Kevin and Vo, Khoa and Yamazaki, Kashu and Le, Ngan},
doi={10.1109/ICIP49359.2023.10222289},
url={https://ieeexplore.ieee.org/document/10222289}
publisher={IEEE International Conference on Image Processing (ICIP)},
pages={3230--3234},
year={2023},
organization={IEEE}
}
Kevin Hyekang Joo - hkjoo@cs.umd.edu