Awesome rPPG

This is a collection of research papers for rPPG. And the repository will be continuously updated to track the frontier of rPPG area.

Welcome to follow and star!

Table of Contents

• Awesome rPPG
  • Table of Contents
  • Overview of rPPG
  • Papers
    • Survey
    • Traditional Methods
    • Supervised Learning
    • Unsupervised Learning
    • Datasets
    • Benchmark

Overview of rPPG

The non-contact measurement of heart rate (HR) and heart rate variability (HRV) has been a hot topic in the field of computer vision and signal processing. The rPPG technology is based on the principle of photoplethysmography (PPG), which uses the light absorption characteristics of blood to measure the heart rate. The rPPG technology can be used in various scenarios, such as health monitoring, emotion recognition, and human-computer interaction.

Papers

Survey

• Video-Based Heart Rate Measurement: Recent Advances and Future Prospects

  • Xun Chen et al, IEEE Transactions on Instrumentation and Measurement (TIM), 2019

• Camera Measurement of Physiological Vital Signs

  • Daniel McDuff et al, ACM Computing Surveys, 2021

• Remote photoplethysmography for heart rate measurement: A review

  • Hanguang Xiao et al, Biomedical Signal Processing and Control, 2023

Traditional Methods

• Remote plethysmographic imaging using ambient light (GREEN)

  • Wim Verkruysse et al, Optics Express, 2008

• Advancements in Noncontact, Multiparameter Physiological Measurements Using a Webcam (ICA)

  • Ming-Zher Poh et al, IEEE Transactions on Biomedical Engineering (TBME), 2010

• Measuring pulse rate with a webcam—a non-contact method for evaluating cardiac activity (PCA)

  • Magdalena Lewandowska et al, 2011 federated conference on computer science and information systems (FedCSIS), 2011

• Robust Pulse Rate From Chrominance-Based rPPG (CHROM)

  • Gerard de Haan and Vincent Jeanne, IEEE Transactions on Biomedical Engineering (TBME), 2013

• Improved motion robustness of remote-PPG by using the blood volume pulse signature (PBV)

  • G de Haan et al, Physiological Measurement, 2014

• A Novel Algorithm for Remote Photoplethysmography: Spatial Subspace Rotation (2SR)

  • Wenjin Wang et al, IEEE Transactions on Biomedical Engineering (TBME), 2015

• Algorithmic Principles of Remote PPG (POS)

  • Wenjin Wang et al, IEEE Transactions on Biomedical Engineering (TBME), 2016

• Local Group Invariance for Heart Rate Estimation from Face Videos in the Wild (LGI)

  • Christian S. Pilz et al, CVPR Workshop, 2018

• Face2PPG: An Unsupervised Pipeline for Blood Volume Pulse Extraction From Faces (OMIT)

  • Constantino Álvarez Casado and Miguel Bordallo López, IEEE Journal of Biomedical and Health Informatics (JBHI), 2023

Supervised Learning

• Visual heart rate estimation with convolutional neural network (HR-CNN)

  • Radim Špetlík et al, In Proceedings of British Machine Vision Conference (BMVC), 2018
  • code

• DeepPhys: Video-Based Physiological Measurement Using Convolutional Attention Networks (DeepPhys)

  • Weixuan Chen and Daniel McDuff, ECCV, 2018

• RhythmNet: End-to-end Heart Rate Estimation from Face via Spatial-temporal Representation (RhythmNet)

  • Xuesong Niu et al, arXiv, 2019
  • code

• Remote Photoplethysmograph Signal Measurement from Facial Videos Using Spatio-Temporal Networks (PhysNet)

  • Zitong Yu et al, BMVC, 2019
  • code

• PulseGAN: Learning to generate realistic pulse waveforms in remote photoplethysmography (PulseGAN)

  • Rencheng Song et al, arXiv, 2020
  • code

• Multi-Task Temporal Shift Attention Networks for On-Device Contactless Vitals Measurement (TS-CAN)

  • Xin Liu et al, NeurIPS, 2020
  • code

• Dual-GAN: Joint BVP and Noise Modeling for Remote Physiological Measurement (Dual-GAN)

  • Hao Lu et al, CVPR, 2021

• PhysFormer: Facial Video-based Physiological Measurement with Temporal Difference Transformer (PhysFormer)

  • Zitong Yu et al, CVPR, 2022
  • code

• PhysFormer++: Facial Video-based Physiological Measurement with SlowFast Temporal Difference Transformer (PhysFormer++)

  • Zitong Yu et al, International Journal of Computer Vision (IJCV), 2023

• EfficientPhys: Enabling Simple, Fast and Accurate Camera-Based Cardiac Measurement (EfficientPhys)

  • Xin Liu et al, WACV, 2023
  • code

• rPPG-MAE: Self-supervised Pre-training with Masked Autoencoders for Remote Physiological Measurement (rPPG-MAE)

  • Xin Liu et al, arXiv, 2023
  • code

• BigSmall: Efficient Multi-Task Learning for Disparate Spatial and Temporal Physiological Measurement (BigSmall)

  • Girish Narayanswamy et al, WACV, 2024

• RhythmFormer: Extracting rPPG Signals Based on Hierarchical Temporal Periodic Transformer (RhythmFormer)

  • Bochao Zou et al, arXiv, 2024
  • code

• RhythmMamba: Fast Remote Physiological Measurement with Arbitrary Length Videos (RhythmMamba)

  • Bochao Zou et al, arXiv, 2024

Unsupervised Learning / Self-supervised Learning

• The Way to my Heart is through Contrastive Learning: Remote Photoplethysmography from Unlabelled Video

  • John Gideon and Simon Stent, ICCV, 2021
  • code

• Contrast-Phys: Unsupervised Video-based Remote Physiological Measurement via Spatiotemporal Contrast (Contrast-Phys)

  • Zhaodong Sun and Xiaobai Li, ECCV, 2022
  • code

• Contrast-Phys+: Unsupervised and Weakly-supervised Video-based Remote Physiological Measurement via Spatiotemporal Contrast (Contrast-Phys+)

  • Zhaodong Sun and Xiaobai Li, IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2024
  • code

• Facial Video-Based Remote Physiological Measurement via Self-Supervised Learning

  • Zijie Yue et al, IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2023

• Non-Contrastive Unsupervised Learning of Physiological Signals from Video (SiNC)

  • Jeremy Speth et al, CVPR, 2023
  • code

• SiNC+: Adaptive Camera-Based Vitals with Unsupervised Learning of Periodic Signals (SiNC+)

  • Jeremy Speth et al, arXiv, 2024

Datasets

• DEAP: A Database for Emotion Analysis ;Using Physiological Signals (DEAP)

  • Sander Koelstra et al, IEEE Transactions on Affective Computing (TAFFC), 2011
  • download

• Non-contact Video-based Pulse Rate Measurement on a Mobile Service Robot (PURE)

  • Ronny Stricker et al, IEEE International Workshop on Robot and Human Communication (ROMAN), 2014
  • download

• Multimodal Spontaneous Emotion Corpus for Human Behavior Analysis (MMSE-HR)

  • Zheng Zhang et al, CVPR, 2016
  • download

• A Reproducible Study on Remote Heart Rate Measurement (COHFACE)

  • G. Heusch et al, arXiv, 2017
  • download

• Unsupervised skin tissue segmentation for remote photoplethysmography", Pattern Recognition Letters (UBFC-RPPG)

  • Serge Bobbia et al, Pattern Recognition Letters, 2017
  • download

• VIPL-HR: A Multi-modal Database for Pulse Estimation from Less-constrained Face Video (VIPL-HR)

  • Xuesong Niu et al, ACCV, 2018
  • download

• The OBF Database: A Large Face Video Database for Remote Physiological Signal Measurement and Atrial Fibrillation Detection (OBF)

  • Xiaobai Li et al, 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), 2018

• Near-Infrared Imaging Photoplethysmography During Driving (MR-NIRP)

  • Ewa M. Nowara et al, IEEE Transactions on Intelligent Transportation Systems (TITS), 2020
  • download

• Evaluation of biases in remote photoplethysmography methods (CMU PPG)

  • Ananyananda Dasari et al, npj digital medicine, 2021
  • download

• UBFC-Phys: A Multimodal Database For Psychophysiological Studies Of Social Stress (UBFC-Phys)

  • Rita Meziati Sabour et al, IEEE Transactions on Affective Computing (TAFFC), 2021
  • download

• Deception Detection and Remote Physiological Monitoring: A Dataset and Baseline Experimental Results (DDPM)

  • Jeremy Speth et al, IEEE Transactions on Biometrics, Behavior, and Identity Science (TBIOM), 2021
  • download

• SCAMPS: Synthetics for Camera Measurement of Physiological Signals (SCAMPS)

  • Daniel McDuff et al, arXiv, 2022
  • download

• Synthetic Generation of Face Videos with Plethysmograph Physiology (UCLA-rPPG, UCLA-synthetic)

  • Zhen Wang et al, CVPR, 2022
  • download

• MMPD: Multi-Domain Mobile Video Physiology Dataset (MMPD)

  • Jiankai Tang et al, EMBC, 2023
  • download

• ReactioNet: Learning High-order Facial Behavior from Universal Stimulus-Reaction by Dyadic Relation Reasoning (BP4D+)

  • Xiaotian Li et al, ICCV, 2023
  • download

• iBVP Dataset: RGB-Thermal rPPG Dataset with High Resolution Signal Quality Labels (iBVP)

  • Jitesh Joshi and Youngjun Cho, Electronics, 2024
  • download

Benchmark

• iPhys: An Open Non-Contact Imaging-Based Physiological Measurement Toolbox (Matlab Toolbox)

  • Daniel McDuff and Ethan Blackford, arXiv, 2019
  • code

• Evaluation of biases in remote photoplethysmography methods (Matlab Toolbox)

  • Ananyananda Dasari et al, npj Digital Medicine, 2021
  • code

• pyVHR: a Python framework for remote photoplethysmography (Python Toolbox)

  • Giuseppe Boccignone et al, PeerJ Computer Science, 2022
  • code

• rPPG-Toolbox: Deep Remote PPG Toolbox (Python Toolbox)

  • Xin Liu et al, NeurIPS 2023
  • code

• Remote Bio-Sensing: Open Source Benchmark Framework for Fair Evaluation of rPPG (Python Toolbox)

  • Dae-Yeol Kim et al, arXiv, 2023
  • code