Skip to content
Using IoTEdge with Cognitive Services Containers to enhance Retro Video Games 🧠+🎮
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.vscode
modules/ScreenshotTranslator Initial Commit May 3, 2019
.gitignore Initial Commit May 3, 2019
README.md Update README.md May 3, 2019
deployment.debug.template.json
deployment.template.json Initial Commit May 3, 2019

README.md

RetroArch-AI-with-IoTEdge 🧠+🎮

Using IoTEdge with Cognitive Services Containers to enhance Retro Video Games

This project uses AI services running side-by-side Retroarch on top of Lakka through the use of containers to allow for interesting interactions with Retroarch in a modular and remotely configurable fashion.

Current Modules

ScreenshotTranslator

Docker Pulls

Demo

Translates screenshots captured in /storage/screenshots/ for display to the framebuffer of fb0 using a local container instance of cognitive-services-recognize-text and a custom ScreenshotTranlator module for IoT Edge

Parameter Description Type
Fontsize Controls the size of the rendered translation font (default: 25) Integer
FontFamily Controls the FontFamily used to render the translation (default: DejaVuSansMono-Bold) string
Language Controls the Language to translate text to (currently only translates from English) string

Ex: To translate to Japanese and display japanese characters, set the desired properties for the module twin of ScreenshotTranslator on the device in question for FontFamily to "TakaoPMincho" and Language to "ja".

FontFamily values are obtained from installed fonts in /usr/share/fonts/truetype/*

Valid values include:

  • DejaVuSans-Bold
  • DejaVuSans
  • DejaVuSansMono-Bold
  • DejaVuSansMono
  • DejaVuSerif-Bold
  • DejaVuSerif
  • TakaoMincho (Japanese)
  • TakaoPMincho (Japanese)
  • NanumGothicBold (Korean)
  • NanumMyeongjoBold (Korean)

Language values can be obtained from the Microsoft Text Translator Language Support Documentation

Installation and Setup

Requires an x64 compatible device with a screen and HDMI out and access to the Cogntive Services Computer Vision Containers Preview.

1 - Download the latest Lakka.img.gz release from this fork of Lakka-LibreELEC and install it by following these instructions.

2 - After installation, configure the device to output the RetroArch UI over HDMI by select the appropriate monitor index in Video Settings, you may also want to configure audio output to route over HDMI as well by configuring the appropriate Audio Device in Audio Settings.

3 - Configure the Wifi settings to connect to an approprite access point in the WiFi-Configuration Interface

4 - SSH into your Lakka Device by following these instructions and execute the following commands to install the Docker Add-On:

cd ~/.kodi/

wget https://github.com/toolboc/Lakka-LibreELEC/releases/download/RetroArch-AI-with-IoTEdge_v1/service.system.docker-8.2.122.zip

unzip service.system.docker-8.2.122.zip

cd ~/.kodi/addons/service.system.docker/system.d/

systemctl enable service.system.docker.service

reboot

Note: Docker will install to the storage partition and is accessible via the path:

/storage/.kodi/addons/service.system.docker/bin/docker

Verify that docker is running with:

export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/storage/.kodi/addons/service.system.docker/bin
docker ps

5 - Create an Azure IoT Hub by following these instructions then create a new IoT Edge Device in the Azure Portal and retrieve the connection string.

6 - SSH into your Lakka Device by following these instructions and verify that docker is running with:

export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/storage/.kodi/addons/service.system.docker/bin
docker ps

Create an instance of an azure-iot-edge-device-container with (be sure to replace with the key obtained in Step 5):

docker run --name edge-device-container --restart always -d --privileged -v /storage/screenshots:/storage/screenshots -v /dev/fb0:/dev/fb0 -e connectionString='<IoTHubDeviceConnectionString>' toolboc/azure-iot-edge-device-container

7 - Install Visual Studio Code onto an available development machine and install the Azure IoT Edge extension for Visual Studio Code.

8 - Clone or download a copy of this repo and open the RetroArch-AI-with-IoTEdge folder in Visual Studio Code. Next, press F1 and select Azure IoT Hub: Select IoT Hub and choose the IoT Hub you created in Step 5, follow the prompts to complete the process.

9 - In VS Code, navigate to the deployment.template.json file and modify the following values:

10 - Create a deployment for the IoT Edge device by right-clicking deployment.template.json and select Generate IoT Edge Deployment Manifest. This will create a file under the config folder named deployment.amd64.json, right-click that file and select Create Deployment for Single Device and select the device you created in Step 5.

Wait a few minutes for the deployment to complete and you should see the logo for the RetroArch Screenshot Translator appear on the main screen of your Lakka machine.

Configure a Hotkey or manually select "Take Screenshot" while playing a game, the translated image is stored to /storage/screenshots/translated/ and displayed on the main screen of the Lakka Device.

You can’t perform that action at this time.