Generates mask data from videos using YOLOv8, Segment Anything, MobileSAM, and NanoSAM. YOLOv8 is used alongside SAM to input a bounding box as a prompt, which allows SAM to segment just the driver's body rather than everything in the frame.
- MobileSAM version available in "fast" branch
- NanoSAM version available in "nano" branch
- Requires installation of YOLOv8 from ultralytics and its dependencies
- Requires installation of Segment Anything, MobileSAM, and NanoSAM and their dependencies
python main.py -i INPUT_VIDEO -o OUTPUT_CSV -f FRAME_SKIP -v VIDEO_PATH
-i INPUT_VIDEO- Path to input video
-o OUTPUT_CSV- Path to CSV file to write coordinates
-f FRAME_SKIP- OPTIONAL FLAG - Used to specify how many frames to skip each iteration
- For example,
-f 6would take every 6th frame of the video instead of all the frames - Defaults to 1 if unspecified, AKA takes every frame as the default
-v VIDEO_PATH- OPTIONAL FLAG - Path to the processed video with the segmentation completed
- If the flag is not specified, a video will not be generated
- Jetson Orin Nano
- JetPack 5.1.3
- Python 3.8
- CUDA 11.4.19
- TensorRT 8.5.2
- 1024-core NVIDIA Ampere architecture GPU with 32 Tensor Cores
- 6-core Arm Cortex-A78AE v8.2 64-bit CPU
- 8GB 128-bit LPDDR5
The following results were from processing 10800 frames of video data.
| Device | Average Time for 1 Frame (ms) | |||
|---|---|---|---|---|
| YOLOv8 | YOLOv8 (TensorRT) | MobileSAM | NanoSAM | |
| Jetson Orin Nano | 40.24 | 37.28 | 345.78 | 61.28 |