🌟WinDB🌟 HMD-Free and Distortion-Free Panoptic Video Fixation Learning (TPAMI 2025)

Guotao Wang ¹,  Chenglizhao Chen ^{2, 6},  Aimin Hao ¹,  Hong Qin ³,  Deng-Ping Fan ^{4, 5}, 

¹ Beihang University  ² China University of Petroleum  ³ Stony Brook University  ⁴ Nankai University 
⁵ Nankai International Advanced Research Institute (SHENZHEN FUTIAN)  ⁶ Sichuan Provincial Key Laboratory of Criminal Examination 

🎣 FishNet Architecture

Fig. The detailed network architecture of our FishNet.

A focuses on performing ERP-based global feature embedding to achieve panoptic perception and avoid visual distortion.
B catches fixation shifting by refocusing the network to avoid the compression problem of shifted fixations in SOTA models.
C makes the network fully aware of the fixation shifting mechanism to ensure that the network is sensitive to fixation shifting.

Fig. Detailed calculation of the spherical distance. Fig. Visualizing of the ``shifting-aware feature enhancing''.

🛠️ Key Steps for FishNet (CODE: https://github.com/guotaowang/FishNet/tree/main)

1. Training Process

   python train.py

2. Inference Process

   python test.py

3. Model Weight

   • Model_best.pth Baidu Netdisk, Google Netdisk (97.9 MB)

4. Results

   • Results are stored in the output directory.

---

📊 Evaluation

1. Score of Each Testing Set Clip

   MatricsOfMyERP.m

2. Score of Entire Testing Set

   MatricsOfMyALLERP.m

---

📜 Citation

If you use WinDB, please cite the following paper:

@article{wang2023windb,
  title={WinDB: HMD-free and Distortion-free Panoptic Video Fixation Learning},
  author={Wang, Guotao and Chen, Chenglizhao and Hao, Aimin and Qin, Hong and Fan, Deng-Ping},
  journal={arXiv preprint arXiv:2305.13901},
  year={2023}
}