Controlling the Baxter robot using Virtual reality device (HTC VIVE). It requires two computers: Windows 10 (Client) and Ubuntu 14.04 (Server). Client computer connects to VR hardware and extracts complete VIVE controller status and sends this data to the Server. The server computer receives the controller information, transforms into Baxter coordinate system, estimates inverse kinematics solutions of Baxter and sends joint increments to Baxter core.
triad_openvr is an open source software built on top of the openvr library. It provides a list of basic functions to access controller information, touch pad coordinates, trigger status, etc. https://github.com/TriadSemi/triad_openvr.git.
In my project, I used triad_openvr library to extract VIVE controller information (Code is present in 'Client_VR' directory)
- Check for the computer hardware requrirements here
- Install Steam VR and Epics Launcher - Unreal Engine.
- Install the headset and controllers properly.
- Download the 'Client_VR' folder present in this repository.
- Run the 'controller_test.py'. If it runs without errors, then VIVE hardware is successfully installed.
- Open 'main.py', change the 'TCP_IP' variable to the IP address of your Ubuntu computer.
- Run 'main.py'
- Ubuntu 14.04 is recommended.
- Install ROS Indigo and Baxter Workstation Setup
- Download the 'Server_Baxter' folder present in this repository. Copy this folder into catkin_ws/src/.
- Open 'main.py', and change the 'TCP_IP' variable to the IP address of Ubuntu computer.
- Run 'main.py'
Note: These instructions does not include live streaming the first view of Baxter on to the VIVE headset. In order to do this, you need to mount a camera on Baxter's head, connect it to Windows computer. Use the Unreal engine (installed in the Windows computer) to live stream webcam in the VIVE headset.