HxI

Human <computer/machine/robot> interfaces (HxI)

Body, Hands, Face, etc. Recognition

OpenPose

• Tracking capabilities: Body, arms, face, multi person tracking
• Info: Project page, ROS wrapper
• Comments: Can be run on a CPU, but utilizing GPU is highly recommended

OpenFace

• Tracking capabilities: Head
• Info: Project page

OpenHeadPose

• Tracking capabilities: Head
• Info: Project page

UltraLeap / LeapMotion

• Tracking capabilities: Hands, arms
• Info: Project page, ROS wrapper
• Comments: Requires dedicated sensor (Leap Motion Controller)

Social LEAP Estimates Animal Poses (SLEAP)

• Tracking capabilities: Animal poses
• Info: Project page

---

• Kinect SDK

• OpenKinect

• Azure Kinect SDK

• Alpha-Pose

• OpenNI/NITE Hands

• FaceReader

• Affdex SDK

• Face API (~12 months free trial)

• dlib

• GazeSense

• EyeTracking

• Algorithms

  • Viola-Jones or Haar classifier
  • cascade classifier

Speech Recognition

VOSK

• Info: Project page, ROS wrapper
• Comments: Doesn't require internet connection

CMUSphinx

• Info: Project page
• Comments: Doesn't require internet connection

Google Speech-to-Text

• Info: Project page, ROS wrapper
• Comments: Requires internet connection and an account. Free up to 30 min of speech processing per month

Microsoft Speech-to-Text

• Info: Project page
• Comments: Requires internet connection. TODO free tier info

Amazon Speech-to-Text

• Info: Project page
• Comments: Requires internet connection. TODO free tier info

Kaldi

• Info: Project page, Tutorial
• Comments:

Julius

• Info: Project page
• Comments:

Natural Language Processing

OpenAI

• Capabilities: Question answering, text generation, etc.
• Info: Project page, Tutorial
• Comments: 18$ free query credit for new users

OpenNLP

• Capabilities: Tokenization, part-of-speech tagging, sentence segmentation, named entity extraction, etc.
• Info: Project page

Natural Language Toolkit (NLTK)

• Capabilities: Tokenization, tagging, stemming, parsing, semantic reasoning, interfaces to corpora and lexical resources such as WordNet
• Info: Project page

MeTA

• Capabilities: Tokenization, part-of-speech tagging, etc.
• Info: Project page

Machine Learning and Signal Processing

• PyTorch (image recognition and language processing)
• YOLO
• OpenVINO
• HuggingFace - repository of ML models
• OpenCV
• ar_track_alvar - AR tag recognition

Sound Processing

• TorchAudio
• Honk (Keyword Spotting)
• openSMILE

Hardware Interfaces

usb_cam

• About: Generic 2D camera driver for ROS applications
• Info: ROS package

realsense_ros

• About: ROS driver for Intel Realsense 3D cameras
• Info: ROS package

UltraLeap / LeapMotion

• About: A dedicated sensor for tracking hands and arms
• Info: Project page, ROS wrapper