ParaGlassMenu is an AR software developed for OHMD (Nreal smartglasses) that enables users to conduct digital interactions subtly using a ring mouse (sanwa) during conversations.
Runze Cai, Nuwan Janaka, Shengdong Zhao, and Minghui Sun. 2023.
ParaGlassMenu: Towards Social-Friendly Subtle Interactions in Conversations.
In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI ’23),
April 23–28, 2023, Hamburg, Germany. ACM, New York, NY, USA, 21 pages.
https://doi.org/10.1145/3544548.3581065
- Runze Cai: runze.cai [at] u.nus.edu
- To run the Python IoT Server, install
Python 3.9and then runpip install -r requirement.txtin theParaGlassMenu - Python IoT Serverfolder. - To use the Unity AR Menu, install
Unity 2019.4.39f1and prepare Nreal Light glasses. - To use the ring mouse, connect it to the Nreal's Computing Unit and install the Key Mapper APK file to the Computing Unit. Then modify the four buttons on the ring mouse to the
[up, down, right, left]arrow keys.
- Modify the
addressandtokensin the code:- Replace the computer's IP address (where the Python server code runs) in the Unity project:
ParaGlassMenu - Unity MenuAssets/NetMQExample/Scripts/HelloRequester.cs. - Replace the IoT devices' address and token in the Python script:
ParaGlassMenu - Python IoT Server/unity_iot_control_sever.py.- You can refer to this link to get the token of the Mi Home IoT devices.
- Download Hue on your smartphone to get the Philips Hue's IP address. [Note: The first connect may require you to click the button on the bridge first]
- Replace the computer's IP address (where the Python server code runs) in the Unity project:
- Open and build the Unity project in Unity's editor. Then, install the APK file of ParaGlassMenu to the Computing Unit for Nreal glasses using ADB.
- To use the music player, create a folder
musicunderParaGlassMenu - Python IoT Serverand add downloaded music to it.