v0.1.0

The first public release of Physiotrack, an open-source Python toolkit for contactless human understanding — turning RGB / RGB-D / thermal video into interpretable physiological and behavioral signals. Developed at the Center for Machine Vision and Signal Processing (CMVS), University of Oulu.

Philosophy: send meaning, not pixels — extract human-state features so downstream systems can reason about states instead of raw video.

⚠️ This is an early 0.x release. The library is under active development and the public API may change before 1.0.

✨ Highlights

• One unified API across every image predictor: configure the model, call .predict(img) (or model(img)), read a structured Result, and draw with result.plot().
• Composable subsystems that work standalone or as an end-to-end pipeline via the Video orchestrator.
• Automatic model management — weights resolve through the Models registry and download on first use (Ultralytics + Hugging Face).

📦 What's included

Subsystem	Capability	Backends
Detection	Multi-person / object boxes	YOLO11, RT-DETR, VR-specific
Tracking	Persistent IDs, occlusion-robust	OC-SORT, ByteTrack, StrongSORT, BoostTrack
Pose 2D	17 (COCO) or 133 (whole-body) keypoints	ViTPose, Sapiens, YOLO11-Pose
Pose 3D	Lift 2D keypoints to 3D over time	MotionBERT, DDHPose
Canonicalization	Viewpoint-invariant 3D pose alignment	3DPCNet, geometric
Segmentation	Pixel-level instance masks	YOLO-Seg, Sapiens, VR-Head
Depth	Monocular dense depth	Depth-Anything-V2 (s/b/l)
Face	Detection + 3D head orientation	YOLO-Face, 6DRepNet360, CMVS-FO-VR
Signals	rPPG (HR/RR) + motion features	POS, CHROM, LGI, OMIT + filters
Views	Bird's-eye floor map, ego-video & depth overlays	—

Inputs: monocular RGB, RGB-D + infrared (Azure Kinect), depth, and thermal video; live cameras, files, or single images.

🚀 Installation

git clone https://github.com/tharindu326/physiotrack.git
cd physiotrack
pip install -e .

Install the PyTorch build matching your platform/CUDA first (requirements.txt pins a CUDA 12.8 build, torch==2.9.1). Model weights are not bundled and download automatically on first use.

🧑‍💻 Quick start

import cv2
import physiotrack as pt

frame = cv2.imread("frame.png")

det    = pt.Detection.Person(conf=0.25, device=0)
result = det.predict(frame)
cv2.imwrite("out.png", result.plot())

pose   = pt.Pose.Person()
people = pose.predict(frame)
wrist  = people[0].keypoints.by_name("left_wrist")
print(wrist.x, wrist.y, wrist.confidence)

End-to-end video pipeline:

from physiotrack import Video, Detection, Pose, Tracker, TrackerConfig, Models

Video(
    source="input.mp4",
    detector=Detection.Person(),
    pose=Pose.Custom(model=Models.Pose.ViTPose.WholeBody.b_wholebody),
    tracker=Tracker(config=TrackerConfig(tracker="ocsort", classes=[0])),
    output_dir="output",
).run(output_video="out.mp4", output_json="out.json")

⚠️ Known limitations

• API is pre-1.0 and subject to change.
• A CUDA GPU is recommended for real-time use; CPU works but is slower.
• Model weights are downloaded on demand and are not part of the package.
• Some subsystems (e.g. parts of signals) are still evolving.

📚 Citation

If you use Physiotrack in your research, please cite the relevant papers listed in the README.

🙏 Acknowledgments

Supported by the University of Oulu and the Research Council of Finland through the 6G Flagship Programme (346208), Profi5 HiDyn (326291), and Profi7 Hybrid Intelligence (352788). Computational resources provided by CSC (IT Center for Science, Finland).

---

Full documentation: see the README. License: GPL-3.0.