notebooks/pyVHR_demo.ipynb

{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"private_outputs":true,"provenance":[{"file_id":"1p-hc4NQyETx-07V7sbvgVOdiiaJzJWCn","timestamp":1673623253938},{"file_id":"1MUMqVpQ1xx6XwS1G4iTHU7lbdMVbHGBJ","timestamp":1640772983263}],"collapsed_sections":["OVvq-D_A2eSQ","TXFwg6twximv","jOASmqeq6HI-","-Z_tKTqNkPaA","Nl0sOh5zOBRl","lthrNMLnGwoE"]},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.8.5"}},"cells":[{"cell_type":"markdown","metadata":{"id":"RndCVqh915_g"},"source":["---\n","# **PyVHR demo**\n","---"]},{"cell_type":"code","metadata":{"id":"nFP37oIXuKQN"},"source":["# -- MAIN IMPORT\n","\n","import pyVHR as vhr\n","import numpy as np\n","\n","# Plotting: set 'colab' for Google Colaboratory, 'notebook' otherwise\n","vhr.plot.VisualizeParams.renderer = 'colab'  # or 'notebook'"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["Below are the possible choices for the datasets.\n","* `PURE`, `UBFC1`, `UBFC2`, `ECG_Fitness_01-1`, `ECG_Fitness_01-2`... `ECG_Fitness_06-1`, `ECG_Fitness_06-2`"],"metadata":{"id":"4cvCF5ktaWIH"}},{"cell_type":"code","metadata":{"id":"YYKTCUkf7Y6l"},"source":["# -- LOAD A DATASET\n","\n","dataset_name = 'UBFC2'                      # the name of the python class handling it \n","video_DIR = '/var/datasets/VHR1/UBFC2/'  # dir containing videos\n","BVP_DIR = '/var/datasets/VHR1/UBFC2/'    # dir containing BVPs GT\n","\n","dataset = vhr.datasets.datasetFactory(dataset_name, videodataDIR=video_DIR, BVPdataDIR=BVP_DIR)\n","allvideo = dataset.videoFilenames\n","\n","# print the list of video names with the progressive index (idx)\n","for v in range(len(allvideo)):\n","  print(v, allvideo[v])"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"AKz1w2i48u-Z"},"source":["# -- PARAMETER SETTING\n","\n","wsize = 8        # seconds of video processed (with overlapping) for each estimate \n","video_idx = 34    # index of the video to be processed\n","fname = dataset.getSigFilename(video_idx)\n","sigGT = dataset.readSigfile(fname)\n","bpmGT, timesGT = sigGT.getBPM(wsize)\n","videoFileName = dataset.getVideoFilename(video_idx)\n","print('Video processed name: ', videoFileName)\n","fps = vhr.extraction.get_fps(videoFileName)\n","print('Video frame rate:     ',fps)"],"execution_count":null,"outputs":[]},{"cell_type":"code","source":["# -- DISPLAY GT SPECTROGRAM\n","\n","sigGT.displaySpectrum()"],"metadata":{"id":"ZdIelmKV-cp0"},"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"bv3FNU0kxLI8"},"source":["# -- DISPLAY VIDEO FRAMES\n","\n","vhr.plot.display_video(videoFileName)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"OVvq-D_A2eSQ"},"source":["# Skin extraction\n","\n","Estract the skin by two methods (using the `SignalProcessing` class):\n","* **convex hull** on landmarks by [MediaPipe Face Mesh](https://google.github.io/mediapipe/solutions/face_mesh) (a face geometry solution that estimates 468 3D face landmarks in real-time)\n","* **face parsing** by the BiSeNet CNN (see [face-parsing.PyTorch](https://github.com/zllrunning/face-parsing.PyTorch))\n","\n","Once the skin is selected, select how to process it: \n","\n","* **Patches**: small facial regions of skin  centered on landmarks (provides multiple estimators)\n","* **Holistic**: convex hull of patches or face parsing CNN (provides a single  estimator)\n","\n","***Note***: `SignalProcessing` is powered by CUDA"]},{"cell_type":"code","metadata":{"id":"0NE9MUZm2jWa"},"source":["sig_extractor = vhr.extraction.SignalProcessing()\n","sig_extractor.display_cuda_device()\n","sig_extractor.choose_cuda_device(0)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"VA8WkKVSaqrh"},"source":["Use convex hull or face parsing to extract the skin"]},{"cell_type":"code","metadata":{"id":"NvtkPciialFx"},"source":["# CPU based\n","# sig_extractor.set_skin_extractor(vhr.extraction.SkinExtractionConvexHull('CPU'))\n","#sig_extractor.set_skin_extractor(vhr.extraction.SkinExtractionFaceParsing('CPU'))\n","\n","# GPU based\n","sig_extractor.set_skin_extractor(vhr.extraction.SkinExtractionConvexHull('GPU'))\n","#sig_extractor.set_skin_extractor(vhr.extraction.SkinExtractionFaceParsing('GPU'))"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"DJNvz6Jqehkj"},"source":["Choose a specific number of frames of the video to process... "]},{"cell_type":"code","metadata":{"id":"2nsze-ygegTB"},"source":["# set the number of seconds (0 for all video)\n","seconds = 0\n","sig_extractor.set_total_frames(seconds*fps)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"m4uYGtL-oYrS"},"source":["### Color-thresholding\n","\n","**OPTIONAL**: Both signal extraction and skin extraction have a color-threshold filter for removing unwanted RGB colors. We can set the RGB threshold interval using theese classes:"]},{"cell_type":"code","metadata":{"id":"Y9q9kmRioXtW"},"source":["vhr.extraction.SkinProcessingParams.RGB_LOW_TH =  0\n","vhr.extraction.SkinProcessingParams.RGB_HIGH_TH = 240\n","\n","vhr.extraction.SignalProcessingParams.RGB_LOW_TH = 0\n","vhr.extraction.SignalProcessingParams.RGB_HIGH_TH = 240"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"cYjkNk5dp8D5"},"source":["## Visualize skin and landmarks \n","\n","* To visualize skin processing intermediate results call `set_visualize_skin_and_landmarks` method.\n","* To retrieve any intermediate result call the methods `get_visualize_skin` and \n","`get_visualize_patches`\n","\n","<img src='https://github.com/giulianogrossi/imgs/blob/main/pyVHR/landmark_on_face.png?raw=true' width=300px >"]},{"cell_type":"code","metadata":{"id":"wyq4D--Pqmey"},"source":["# -- SET VISUALIZATION MODE \n","sig_extractor.set_visualize_skin_and_landmarks(\n","      visualize_skin=True, \n","      visualize_landmarks=True, \n","      visualize_landmarks_number=True, \n","      visualize_patch=True)"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"K9HozzlZm0iH"},"source":["# -- DEFINE A LANDMARK SUBSET\n","\n","# choose predefined...\n","landmarks = vhr.extraction.MagicLandmarks.cheek_left_top +\\\n","                   vhr.extraction.MagicLandmarks.forehead_center +\\\n","                   vhr.extraction.MagicLandmarks.forehoead_right +\\\n","                   vhr.extraction.MagicLandmarks.cheek_right_top +\\\n","                   vhr.extraction.MagicLandmarks.forehead_left +\\\n","                   vhr.extraction.MagicLandmarks.nose \n","\n","# ... or sample the face by 100 equispaced landmarks\n","landmarks = [2, 3, 4, 5, 6, 8, 9, 10, 18, 21, 32, 35, 36, 43, 46, 47, 48, 50, 54, \\\n","             58, 67, 68, 69, 71, 92, 93, 101, 103, 104, 108, 109, 116, 117, \\\n","             118, 123, 132, 134, 135, 138, 139, 142, 148, 149, 150, 151, 152, 182, 187, 188, 193, 197, 201, 205, 206, 207, \\\n","             210, 211, 212, 216, 234, 248, 251, 262, 265, 266, 273, 277, 278, 280, \\\n","             284, 288, 297, 299, 322, 323, 330, 332, 333, 337, 338, 345, \\\n","             346, 361, 363, 364, 367, 368, 371, 377, 379, 411, 412, 417, 421, 425, 426, 427, 430, 432, 436]\n","\n","\n"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"IokA_s8Q9zRG"},"source":["# -- VISUALIZE LANDMARK SUBSET\n","\n","print('Num landmarks: ', len(landmarks))\n","vhr.plot.visualize_landmarks_list(landmarks_list=landmarks)"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"k-e9gio0FIDF"},"source":["# -- SET THE LANDMARK LIST\n","sig_extractor.set_landmarks(landmarks)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"TXFwg6twximv"},"source":["# ROI processing and RGB computation\n","\n","Choose how to extract the RGB signal from ROI:\n","\n","* **Holistic** mean\n","* **Patches** mean\n","\n","Patches are square (with a fixed edge for all) or rectangular (with xy_dimension for each region)."]},{"cell_type":"markdown","metadata":{"id":"jOASmqeq6HI-"},"source":["## Holistic extraction"]},{"cell_type":"code","metadata":{"id":"QXcR6pc49HL_"},"source":["# -- HOLISTIC EXTRACTION\n","hol_sig = sig_extractor.extract_holistic(videoFileName)\n","print('Size: (#frames, #landmarks, #channels) = ',hol_sig.shape)"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"AE04zo6Xf2xl"},"source":["# -- INTERACTIVE VISUALIZATION OF PATCHES\n","visualize_skin_coll = sig_extractor.get_visualize_skin()\n","print('Number of frames processed: ',len(visualize_skin_coll))\n","vhr.plot.interactive_image_plot(visualize_skin_coll,1.0)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"Iv9Csst87sHb"},"source":["## Patches extraction"]},{"cell_type":"code","metadata":{"id":"fxzLGh5rfYPC"},"source":["# -- PATCHES EXTRACTION\n","sig_extractor.set_square_patches_side(80.0)\n","patch_sig = sig_extractor.extract_patches(videoFileName, \"squares\", \"mean\")\n","print('Size: (#frames, #landmarks, #channels) = ', patch_sig.shape)"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"Q6SxcW_bi-Sf"},"source":["# -- INTERACTIVE VISUALIZATION OF PATCHES\n","visualize_patches_coll = sig_extractor.get_visualize_patches()\n","print('Number of frames processed: ',len(visualize_patches_coll))\n","vhr.plot.interactive_image_plot(visualize_patches_coll,1.0)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"6fnzk_QEyH9L"},"source":["## Signal windowing\n","\n","Windowing means to split a video into a set of strided and overlapped windows of frames. For each window the RGB signal is estracted by averaging over pixels in holistic (all skin pixels) or local (averaging on patches) fashion. Shapes are `(rgb_channels, #frames)` and `(#landmarks, rgb_channels, #frames)` respectively. "]},{"cell_type":"markdown","metadata":{"id":"-9itXtb-_eBn"},"source":["### Holistic\n"]},{"cell_type":"code","metadata":{"id":"cYi_G8rzgfHB"},"source":["# -- WINDOWING OF RGB SIGNALS (HOLISTIC)\n","windowed_hol_sig, timesES = vhr.extraction.sig_windowing(hol_sig, wsize, 1, fps)\n","print('Num windows: ',len(windowed_hol_sig))\n","print('Num channels and window length: ', windowed_hol_sig[0].shape)"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"zPXLgInXb2tl"},"source":["# -- PLOT A WINDOW (randomly chosen)\n","w = np.random.randint(0, len(windowed_hol_sig))   # window number\n","vhr.plot.visualize_windowed_sig(windowed_hol_sig, w)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"BZ0QLyLqAqXy"},"source":["### Patches\n"]},{"cell_type":"code","metadata":{"id":"2GMFO4IM_YGm"},"source":["# -- WINDOWING OF RGB SIGNALS ON PATCHES \n","windowed_patch_sig, timesES = vhr.extraction.sig_windowing(patch_sig, wsize, 1, fps)\n","print('Num windows: ',len(windowed_patch_sig))\n","print('Num channels and window length: ', windowed_patch_sig[0].shape)"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"oeyvKWW8_YGn","scrolled":true},"source":["# -- PLOT A WINDOW (randomly chosen)\n","w = np.random.randint(0, len(windowed_patch_sig))  # window number\n","vhr.plot.visualize_windowed_sig(windowed_patch_sig, w)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"-Z_tKTqNkPaA"},"source":["# Pre-filtering\n","\n","The implemented (standard) filters are:\n","\n","* `rgb_filter_ths`: color threshold filter that filters out signals that, in at least one frame of the window, are outside the rgb colors interval `[(LOW, LOW, LOW), (HIGH, HIGH, HIGH)]` where `LOW` is the dictionary parameter `RGB_LOW_TH`, and `HIGH` is `RGB_HIGH_TH` (we suggest to always use this filter before applying a BVP method)\n","* `detrend`: apply detrend to the signal\n","* `sg_detrend`: apply detrend to the signal, i.e. remove the low-frequency components with the low-pass filter developed by Savitzky-Golay\n","* `zscore`: apply z-score to the signal\n","* `BPfilter`: apply Butterworth band-pass filter to the signal"]},{"cell_type":"markdown","metadata":{"id":"MYYspTp_FUgH"},"source":["### Holistic"]},{"cell_type":"code","metadata":{"id":"47GUg8tvvEz2"},"source":["# -- APPLY TRESHOLDING ON RGB COLORS (suggested)\n","\n","filtered_windowed_hol_sig = vhr.BVP.apply_filter(windowed_hol_sig, vhr.BVP.rgb_filter_th, fps=fps, params={'RGB_LOW_TH': 0, 'RGB_HIGH_TH': 230})\n","\n","print('Num windows: ', len(filtered_windowed_hol_sig))\n","print('Win size: (#signals, #channels, #frames) = ', filtered_windowed_hol_sig[0].shape)"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"E90CXeY2kPH8"},"source":["# -- SELECT THE FILTER CASCADE\n","\n","filtered_windowed_hol_sig = vhr.BVP.apply_filter(filtered_windowed_hol_sig, vhr.BVP.BPfilter, params={'order':6,'minHz':0.65,'maxHz':4.0,'fps':fps})\n","#filtered_windowed_hol_sig = vhr.BVP.apply_filter(filtered_windowed_hol_sig, vhr.BVP.detrend)\n","#filtered_windowed_hol_sig = vhr.BVP.apply_filter(filtered_windowed_hol_sig, vhr.BVP.sg_detrend)\n","#filtered_windowed_hol_sig = vhr.BVP.apply_filter(filtered_windowed_hol_sig, vhr.BVP.zscore)\n","#filtered_windowed_hol_sig = vhr.BVP.apply_filter(filtered_windowed_hol_sig, vhr.BVP.zeromean)\n","print('Num windows: ', len(filtered_windowed_hol_sig))\n","print('Win size: (#signals, #channels, #frames) = ', filtered_windowed_hol_sig[0].shape)"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"KrkIidXEIH9P"},"source":["# -- PLOT A WINDOW (randomly chosen)\n","w = np.random.randint(0, len(windowed_hol_sig))  # window number\n","vhr.plot.visualize_windowed_sig(filtered_windowed_hol_sig, w)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"1Wnlf0ZRFb8j"},"source":["### Patches"]},{"cell_type":"code","metadata":{"id":"izOdRE6dse5Q"},"source":["# -- APPLY TRESHOLDING ON RGB COLORS (suggested)\n","\n","filtered_windowed_patch_sig = vhr.BVP.apply_filter(windowed_patch_sig, vhr.BVP.rgb_filter_th, params={'RGB_LOW_TH': 0, 'RGB_HIGH_TH': 230})\n","print('Num windows: ', len(filtered_windowed_patch_sig))\n","print('Win size: (#landmarks, #channels, #frames) = ', filtered_windowed_patch_sig[1].shape)"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"STAe3r7lFktH"},"source":["# -- SELECT THE FILTER CASCADE\n","\n","filtered_windowed_patch_sig = vhr.BVP.apply_filter(filtered_windowed_patch_sig, vhr.BVP.BPfilter, params={'order':6,'minHz':0.75,'maxHz':4.0,'fps':fps})\n","#filtered_windowed_patch_sig = vhr.BVP.apply_filter(filtered_windowed_patch_sig, vhr.BVP.sg_detrend)\n","#filtered_windowed_patch_sig = vhr.BVP.apply_filter(filtered_windowed_patch_sig, vhr.BVP.detrend, params={'detLambda':100}) \n","#filtered_windowed_patch_sig = vhr.BVP.apply_filter(filtered_windowed_patch_sig, vhr.BVP.zscore)\n","#filtered_windowed_patch_sig = vhr.BVP.apply_filter(filtered_windowed_patch_sig, vhr.BVP.zeromean)\n","print('Num windows: ', len(filtered_windowed_patch_sig))\n","print('Win size: (#landmarks, #channels, #frames) = ', filtered_windowed_patch_sig[0].shape)"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"DRsqqG9dH4BE"},"source":["# -- PLOT A WINDOW (randomly chosen)\n","\n","w = np.random.randint(0, len(windowed_patch_sig))  # window number\n","vhr.plot.visualize_windowed_sig(filtered_windowed_patch_sig, w)\n"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"72zO3M0wDOrQ"},"source":["#Method: BVP extraction\n","\n","To extract the BVP signal call the function `RGB_sig_to_BVP` with the following parameters:\n","\n","\n","*   `filt_windowed_sig`: the list of windows\n","*   `fps`: frame rate\n","*   `device_type`: `cuda`, `cpu`, `torch`\n","*   `method`: method function that supports method_type device\n","*   `params`: dictionary of parameters needed by the method ( default is {}).\n","\n","Core methods implemented within pyVHR:\n","* **Methods**: `cpu_CHROM`, `cupy_CHROM`, `torch_CHROM`, `cpu_LGI`, `cpu_POS`, `cupy_POS`, `cpu_PBV`, `cpu_PCA`, `cpu_GREEN`, `cpu_OMIT`, `cpu_ICA`, `cpu_SSR`\n","\n","\n","***Note***: pyVHR contains many methods, but you can also use a custom method. Remember that it must accept a numpy.ndarray with shape (num_estimators, channels, num_frames) and return a numpy.ndarray with shape (num_estimators, num_frames)\n"]},{"cell_type":"markdown","metadata":{"id":"IEYpTmpiFOtd"},"source":["### Holistic"]},{"cell_type":"code","metadata":{"id":"sySCrAinghKt","scrolled":true},"source":["# -- APPLY A METHOD TO EXTRACT BVP\n","\n","from pyVHR.BVP import *\n","\n","hol_bvps = RGB_sig_to_BVP(windowed_hol_sig, fps, device_type='cpu', method=cpu_CHROM)\n","#hol_bvps = RGB_sig_to_BVP(windowed_hol_sig, fps, device_type='cuda', method=cupy_CHROM)\n","#hol_bvps = RGB_sig_to_BVP(windowed_hol_sig, fps, device_type='torch', method=torch_CHROM)\n","#hol_bvps = RGB_sig_to_BVP(windowed_hol_sig, fps, device_type='cuda', method=cupy_POS, params={'fps':fps})\n","#hol_bvps = RGB_sig_to_BVP(windowed_hol_sig, fps, device_type='cpu', method=cpu_POS, params={'fps':fps})\n","#hol_bvps = RGB_sig_to_BVP(windowed_hol_sig, fps, device_type='cpu', method=cpu_LGI)\n","#hol_bvps = RGB_sig_to_BVP(windowed_hol_sig, fps, device_type='cpu', method=cpu_GREEN)\n","#hol_bvps = RGB_sig_to_BVP(windowed_hol_sig, fps, device_type='cpu', method=cpu_ICA, params={'component':'all_comp'})\n","\n","print('Number of windows: ', len(hol_bvps))\n","print('Number of estimators and number of number of frames in a windows: ', hol_bvps[0].shape)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"2OvQ8Au_1eLL"},"source":["*bvps* is a list of length num_windows of numpy.ndarray with shape (num_estimators,num_frames)"]},{"cell_type":"code","metadata":{"id":"M_lZaw95mBtA"},"source":["# -- PLOT A WINDOW (randomly chosen)\n","\n","w = np.random.randint(0, len(windowed_hol_sig))  # window number\n","vhr.plot.visualize_BVPs(hol_bvps, w)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"76QJ_Rs8FR8d"},"source":["###Patches\n","\n"]},{"cell_type":"code","metadata":{"scrolled":true,"id":"lABT_DrHDfVr"},"source":["# -- APPLY A METHOD TO EXTRACT BVP\n","\n","from pyVHR.BVP import *\n","\n","patch_bvps = RGB_sig_to_BVP(filtered_windowed_patch_sig, fps, device_type='cpu', method=cpu_CHROM)\n","#patch_bvps = RGB_sig_to_BVP(filtered_windowed_patch_sig, fps, device_type='cuda', method=cupy_CHROM)\n","#patch_bvps = RGB_sig_to_BVP(filtered_windowed_patch_sig, fps, device_type='torch', method=torch_CHROM)\n","#patch_bvps = RGB_sig_to_BVP(filtered_windowed_patch_sig, fps, device_type='cuda', method=cupy_POS, params={'fps':fps})\n","#patch_bvps = RGB_sig_to_BVP(filtered_windowed_patch_sig, fps, device_type='cpu', method=cpu_POS, params={'fps':fps})\n","#patch_bvps = RGB_sig_to_BVP(filtered_windowed_patch_sig, fps, device_type='cpu', method=cpu_LGI)\n","#patch_bvps = RGB_sig_to_BVP(filtered_windowed_patch_sig, fps, device_type='cpu', method=cpu_GREEN)\n","#patch_bvps = RGB_sig_to_BVP(filtered_windowed_patch_sig, fps, device_type='cpu', method=cpu_ICA, params={'component':'second_comp'})\n","\n","print('Number of windows: ', len(patch_bvps))\n","print('Number of estimators and number of number of frames in a windows: ', patch_bvps[0].shape)"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"gtvJWZqgDfVr"},"source":["# -- PLOT A WINDOW (randomly chosen)\n","w = np.random.randint(0, len(windowed_patch_sig))  # window number\n","vhr.plot.visualize_BVPs(patch_bvps, w)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"O2abphyGZjbA"},"source":["# Post Filtering\n","\n","As with prefiltering, we can apply all the filters showed before also to the *BVP*. "]},{"cell_type":"markdown","metadata":{"id":"8kKjt_liFyBP"},"source":["###Holistic\n"]},{"cell_type":"code","metadata":{"id":"Ovv6a1hTZlfr"},"source":["# -- APPLY BPFILTER TO BVP WINDOWED \n","\n","hol_bvps = vhr.BVP.apply_filter(hol_bvps, BPfilter, params={'order':6,'minHz':0.65,'maxHz':4.0,'fps':fps})\n","print('Num windows: ', len(hol_bvps))"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"bCFL1QM1lnV4"},"source":["# -- PLOT A WINDOW (randomly chosen)\n","wind = np.random.randint(0, len(windowed_hol_sig))  # window number\n","vhr.plot.visualize_BVPs(hol_bvps, wind)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"DDcy6Y4zF0-g"},"source":["###Patches\n"]},{"cell_type":"code","metadata":{"id":"WYpLwScIDfVr"},"source":["# -- APPLY BPFILTER TO BVP WINDOWED PATCHES\n","\n","patch_bvps = vhr.BVP.apply_filter(patch_bvps, BPfilter, params={'order':6,'minHz':0.75,'maxHz':4.0,'fps':fps})\n","print('Num windows: ', len(patch_bvps))\n","print('Win size: (#landmarks, #frames) = ', patch_bvps[0].shape)"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"o9dIB17SDfVr"},"source":["# -- PLOT A WINDOW (randomly chosen)\n","\n","w = np.random.randint(0, len(windowed_patch_sig))  # window number\n","vhr.plot.visualize_BVPs(patch_bvps, w)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"Nl0sOh5zOBRl"},"source":["##BVP spectrum\n","\n","BVP spectrum analysis via PSD for holistic and patches approaches."]},{"cell_type":"markdown","metadata":{"id":"npOb_VhBGvuX"},"source":["###Holistic\n"]},{"cell_type":"code","metadata":{"id":"GpZ4EvdcOmfi"},"source":["w = np.random.randint(0, len(windowed_hol_sig))  # window number\n","vhr.plot.visualize_BVPs_PSD(hol_bvps, w, fps)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"lthrNMLnGwoE"},"source":["###Patches\n"]},{"cell_type":"code","metadata":{"id":"8pty64CpG0rb"},"source":["wind = np.random.randint(0,len(patch_bvps))  # window number\n","vhr.plot.visualize_BVPs_PSD(patch_bvps, wind, fps, maxHz=10)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"2kGmDLaA1BxR"},"source":["# BMP estimation \n","\n","This function process all the windows and all the estimators (one for holistic and many for patches), and returns a list of numpy ndarray with shape (num_estimators,)."]},{"cell_type":"markdown","metadata":{"id":"aDD-I0d8F84j"},"source":["###Holistic\n"]},{"cell_type":"code","metadata":{"id":"IT5Q0v7qgjrh"},"source":["# -- BPM ESTIMATION \n","\n","#hol_bpmES = vhr.BPM.BVP_to_BPM(hol_bvps, fps)       # CPU version\n","hol_bpmES = vhr.BPM.BVP_to_BPM_cuda(hol_bvps, fps)  # CUDA version"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"4WMWwvD0lU0o"},"source":["### Patches - medians"]},{"cell_type":"code","metadata":{"id":"cdJu8gZRfwxn"},"source":["# -- BPM ESTIMATION BY PATCHES\n","#patch_bpmES = vhr.BPM.BVP_to_BPM(patch_bvps, fps)          # CPU version\n","\n","patch_bpmES = vhr.BPM.BVP_to_BPM_cuda(patch_bvps, fps)    # CUDA version"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"f8szh546IT1h"},"source":["# -- MEDIANS OF BPMS\n","\n","patch_median_bpmES, MAD = vhr.BPM.BPM_median(patch_bpmES)"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"PTOWSMC3JIUV"},"source":["# -- VISUALIZE ALL BPMs AND MEDIANS\n","vhr.plot.visualize_multi_est_BPM_vs_BPMs_list([patch_bpmES, timesES], [[patch_median_bpmES, timesES, \"medianES\"],[bpmGT, timesGT, \"GT\"]])"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"-Rw9CX-NGQCs"},"source":["### Patches - PSD clustering\n"]},{"cell_type":"code","metadata":{"id":"SlWDlTPPDfVt"},"source":["# -- BPM ESTIMATION BY PSD CUMUL\n","\n","ma = vhr.extraction.MotionAnalysis(sig_extractor, wsize, fps)\n","psd_bpmES = vhr.BPM.BPM_clustering(ma, patch_bvps, fps, wsize, movement_thrs=[10, 5, 1], opt_factor=0.5)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"OSg5KALx2xpH"},"source":["# BPM vs GT ANALYSIS\n","\n","Error computation and visualization "]},{"cell_type":"markdown","metadata":{"id":"BDKRn0jbHDIp"},"source":["###Holistic\n"]},{"cell_type":"code","metadata":{"id":"BfvIniEcHBBs"},"source":["# -- PRINT ERRORS USING METRICS: RMSE, MAE, MAX, PCC, CCC, SNR\n","\n","from pyVHR.utils.errors import getErrors, printErrors, displayErrors\n","\n","RMSE, MAE, MAX, PCC, CCC, SNR = getErrors(hol_bvps, fps, hol_bpmES, bpmGT, timesES, timesGT)\n","printErrors(RMSE, MAE, MAX, PCC, CCC, SNR)\n","displayErrors(hol_bpmES, bpmGT, timesES, timesGT)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"nw4eCc7QHCwN"},"source":["###Patches - medians\n"]},{"cell_type":"code","metadata":{"id":"HU1Hg6E9gngX"},"source":["# -- PRINT ERRORS USING METRICS: RMSE, MAE, MAX, PCC, CCC, SNR\n","\n","from pyVHR.utils.errors import getErrors, printErrors, displayErrors\n","\n","\n","RMSE, MAE, MAX, PCC, CCC, SNR = getErrors(patch_bvps, fps, patch_median_bpmES, bpmGT, timesES, timesGT)\n","printErrors(RMSE, MAE, MAX, PCC, CCC, SNR)\n","displayErrors(patch_median_bpmES, bpmGT, timesES, timesGT)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"a946Rp84lq2K"},"source":["###Patches - PSD clustering"]},{"cell_type":"code","metadata":{"id":"IFAZxCq0lyRR"},"source":["# -- PRINT ERRORS USING METRICS: RMSE, MAE, MAX, PCC, SNR\n","from pyVHR.utils.errors import getErrors, printErrors, displayErrors\n","\n","RMSE, MAE, MAX, PCC, CCC, SNR = getErrors(patch_bvps, fps, psd_bpmES, bpmGT, timesES, timesGT)\n","printErrors(RMSE, MAE, MAX, PCC, CCC, SNR)\n","displayErrors(psd_bpmES, bpmGT, timesES, timesGT)"],"execution_count":null,"outputs":[]}]}