This project contains the source code to compute the Volumetric Facial Palsy Grading
published at the MIE-2022. You can find the accompanying paper here.
If you use any part of the code for your own research or development, please use the provided citation at the end of the document.
The code features several processing steps which can be used independently from each other:
- Loading of different facial landmark formats:
- Loading of different point-cloud/mesh formats:
.ply
.obj
.wrl
- Computation of
Radial Curves
using the 3D landmarks and the point clouds - Computation of the volumetric score using the
z
point-wise mean difference
All calculations are done in numpy
, scipy
, and igraph
on the CPU.
We utilize multiprocessing during the extraction and processing of the Radial Curves
to achieve almost real-time results.
Note: The system has been developed and extensively been tested only on Linux based systems. For Windows
and MacOS
based system please open an issue of problems occur.
We assume you have a proper python installation on your system and using either a virtual environment or anaconda.
One can create a conda
environment using:
conda create -n corc-test python=3.9 pip -y
To install the corc
please copy the repository into a directory of your choise.
git clone git@github.com:cvjena/corc.git
Inside the folder please run
pip install .
Now the the package should be available in your python installation via
import corc
The following sub-modules are available for usage inside the package
from corc import core, grading, landmarks, utils
This is a minimal example on how to use the corc
library to compute the Radial Curves
from a given 3D landmark and a point-cloud file.
For extended usage please refer to the doc-strings of the functions.
The algorithm usage can be fine-tuned using the defined keyword arguments.
import pathlib
import numpy as np
from corc import core, grading, landmarks, utils
# we assume you use the basic 68 landmark system in this example
lm_file = pathlib.Path("path/to/your/landmarks_file.csv")
pc_file = pathlib.Path("path/to/your/pointcloud_file.obj")
# the loaded files will be in the form of a numpy array.
lm: np.ndarray = utils.load_landmarks(lm_file)
pc: np.ndarray = utils.load_pointcloud(pc_file)
# the landmarks have to forwarded to the correct Landmark class
# this class handles the correct access of the specific landmark features
# like eyes, nose-tip, mouth
radial_curves: np.ndarray = core.compute_corc(
point_cloud=pc,
landmarks=landmarks.Landmarks68(lm),
n_curves=128,
n_points=128,
)
# the computed radial curves are also in the form of the numpy array
# in this example they have the form of 128x128x3
grad: float = grading.volume_grading(radial_curves, n_curves=128, n_points=128)
import pathlib
import numpy as np
from corc import core, grading, landmarks, utils
# we assume you use the basic 68 landmark system in this example
lm_file = pathlib.Path("path/to/your/landmarks_file.csv")
pc_file = pathlib.Path("path/to/your/pointcloud_file.obj")
# the loaded files will be in the form of a numpy array.
lm: np.ndarray = utils.load_landmarks(lm_file)
pc: np.ndarray = utils.load_pointcloud(pc_file)
# the landmarks have to forwarded to the correct Landmark class
# this class handles the correct access of the specific landmark features
# like eyes, nose-tip, mouth
radial_curves: np.ndarray = core.compute_corc(
point_cloud=pc,
landmarks=landmarks.Landmarks68(lm),
n_curves=128,
n_points=128,
)
curves_l, curves_r = corc.volume.split_left_right(curves, n_curves=n_curves)
segment_point, center_point = corc.volume.compute_lower_curves_center(curves_l, factor=1.0)
points_r, triangles_r = corc.volume.make_mesh(curves_r, segment_point, center_point, right=True)
points_l, triangles_l = corc.volume.make_mesh(curves_l, segment_point, center_point, right=False)
mesh_l, volume_l = corc.volume.compute_volume(points_l, triangles_l)
mesh_r, volume_r = corc.volume.compute_volume(points_r, triangles_r)
print("L Volume:", volume_l)
print("R Volume", volume_r)
If you use this work please use the following bibtext entry:
@inproceedings{Buechner22:MIE,
type = {inproceedings},
key = {Buechner22:MIE},
title = {Automatic Objective Severity Grading of Peripheral Facial Palsy Using 3D Radial Curves Extracted from Point Clouds},
author = {Tim Büchner and Sven Sickert and Gerd F. Volk and Orlando Guntinas-Lichius and Joachim Denzler},
booktitle = {Challenges of Trustable AI and Added-Value on Health},
series = {Studies in Health Technology and Informatics},
volume = {294},
year = {2022},
note = {},
publisher = {IOS Press},
pages = {179-183},
doi = {10.3233/SHTI220433},
url = {https://ebooks.iospress.nl/volumearticle/59591},
abstract = {Peripheral facial palsy is an illness in which a one-sided ipsilateral paralysis of the facial muscles occurs due to nerve damage. Medical experts utilize visual severity grading methods to estimate this damage. Our algorithm-based method provides an objective grading using 3D point clouds. We extract from static 3D recordings facial radial curves to measure volumetric differences between both sides of the face. We analyze five patients with chronic complete peripheral facial palsy to evaluate our method by comparing changes over several recording sessions. We show that our proposed method allows an objective assessment of facial palsy.},
groups = {facialpalsy},
langid = {english},
}