The VideoProcessMining project provides a general approach to extract process-mining-conform event logs from unstructured video data. It targets custom datasets and spatio-temporal activity recognition.
It is an implementation of the reference architecture presented in "Shedding light on blind spots – Developing a reference architecture to leverage video data for process mining".
- Get latest version on ScienceDirect (works until June 17,2022)
- See article in Decision Support Systems
- arxiv
The VideoProcessMining project uses several open source computer vision projects.
- Detectron2 (License): For object detection
- Deep Sort with PyTorch (License): For object tracking
- PySlowFast (License): For activity recognition
- PM4Py (License): For XES event log generation
- Crepe Dataset from the paper STARE: Spatio-Temporal Attention Relocation for Multiple Structured Activities Detection
- imutils (License) for fast video frame reading
The VideoProcessMining project is released under the GNU General Public License v3.0.
We stated our changes in the modified files and included the original licenses in the licenses directory.
The VideoProcessMining project comprises a GUI that automates and facilitates several tasks:
- Preprocessing of custom video datasets for spatio-temporal activity recognition
- Training and testing of activity recognition models
- Extraction of process-mining-conform event logs from unstructured video data (demo)
Please find installation instructions in INSTALL.md.
You may follow the instructions in DATASET.md to prepare your custom dataset.
After preparing your dataset, you can follow the instructions in INSTRUCTIONS.md to train and test your models.
You can also extract information from video data.
If you are interested in reproducing our results, follow the instructions in CREPE_DEMO.md.