Image/Audio sensing and decision making for Kaia.ai robots

Implements this functionality:

• WebRTC setup
  • video, audio and data channels
  • a WebRTC-to-ROS2 bridge
• Web server setup
  • a website for robot's smartphone/tablet (TODO)
  • SSL provider
• Robot face controller (TODO)
• Image processing (TODO)
• Audio processing (TODO)
• Decision making (TODO)

PC Setup Instructions

• Windows PC setup instructions here

• Windows PC setup instructions video.

• Linux PC setup

  • install Docker

• run docker pull kaiaai/kaiaai:humble or docker pull kaiaai/kaiaai:iron

Bringup Instructions

# On your host PC, launch Kaia.ai Docker image
docker run --name makerspet -it --rm -p 8888:8888/udp -p 4430:4430/tcp -e DISPLAY=host.docker.internal:0.0 -e LIBGL_ALWAYS_INDIRECT=0 kaiaai/kaiaai:humble

# Launch SSL proxy
ros2 run kaiaai_python ssl_proxy.sh

# On your host PC, open one more bash session
docker exec -it makerspet bash

# Launch web server
ros2 run kaiaai_python web_server

# Open a browser on your host PC
#   Navigate to https://localhost:4430
# Open a browser on your smartphone/tablet
#   Navigate to https://YOUR_HOST_PC_IP:4430
# Click YES/ALLOW/OK on your browser prompt to start video/audio/data WebRTC streaming
#   A robot's animated interactive face should appear (TODO)

Release notes

v0.1.0 in debug

• set up aiohttp web server with aiortc WebRTC
• added WebRTC-to-ROS2 bridge
  • browser webcam video streams over WebRTC, gets published to ROS2
• ROS2 style web server parameters
  • logging from within web server to ROS2 does not work
• enabled SSL support using temporary certs
  • SSL certs replace launching a separate ssl-proxy
• web server launch file
• added kaia-face.js example
• added web cam FPS selection