-
Notifications
You must be signed in to change notification settings - Fork 0
2 System Design
(Author: Patrick Pogscheba, date: 12/2023)
The cubes system was developed as a mixture of client-server and peer-to-peer architecture. The main protagonsits are the cubes. A central server is installed for system & app management and to host central services. To enable a game master to run a cube session there should be a tablet to provide the web interface for system and app control.
There were defined some requirements for cube usage while conceptualizing some possible apps with some designbers and the teams of FFT and MiReVi:
- cubiq form factor
- plain display (no touch)
- audio
- interaction by motion
- rotation steps (small amount of rotation in little time)
- rotation impulses (small amount of rotation in one direction followed by restore old orientation)
- translation steps (small amount of translation in little time)
- moving detection
- orientation
- detect nearby cubes (neighbourhood)
- global position (occlusion resistent)
- communication
- hosting apps with different contents (media display, games, ... ?)
- control of the game by a game master
- easy development & deployment
To fulfill the requierements a basic cube housing was built with alau frames and wood plates. The size is ~60cm per side. A quadratic display is placed on top. inside the housing a small form factor computer is placed along with a sensorboard for neighbourhood and motion recognition. A digital amplifier and a mono loudspeaker was placed, too. The system is powered by a battery pack.
Some more details on the housing and its interior components is shown here: Blueprints
To provide positional tracking a UWB-Positioning System is used. The accuracy isn't as good as other technologies but occlusion by multiple users in front of the cubes on different sides isn't a big concern with UWB. It is not really important which system is used, For the FFT installation we used the Ubitrack system which comes with a linux VM that can be deployed on the server. A seperate network is installed just for the tracking.
The server is a small form factor NUC-like computer with 2 network slots (one for tracking and one for cubes LAN). For the tracking at least two cores and 4GB RAM are needed. To have some power for more tasks we chose 4 cores and 8GB RAM with a 256GB SSD HD.
A standard tablet can be used to give the gamemaster access to the web frontends. A web browser should be installed.
The cubes network is provided by a Wifi-Router via Wifi to the mobile cubes and by wire to the server. The server's second interface is connected to the tracking (PoE-Switch connected to the tracking anchors). The Wifi router should also be connected to the internet.
The cubes system can be splitted in app usage & development.
When a app session is planned for the cubes, a qualified person should use the provided tablet with a web frontend for app & system control. Here the active cubes can be observed and controlled (e.g. observe the current cube system state, control volume or system tasks). Also a list of deployed apps is shown. When an app is started an optional app-dedicated web interface is opened (Gamemaster App). The cubes start the selected app (CubeApp). On the dedicated frontend the app on the cubes can be controlled (control the app, modify settings, do some phase logic, etc.). When the app is finished the app can be closed by the gamemaster. All the cubes closes the running app and return to idle state.
To develop a cube app the basic functionality (communications and interfaces) should be imported by libraries into the project. Initial testing should be done locally, later integration testing can be done in the production system. The developer should implement a cube app (the game) along with a control app (web frontend). When everything is done the deployment of the whole app should be straight forward by just pushing the app to the server with some basic descriptions and settings so the app is automatically listed in the management app and synced to the cubes.
After conceptuialization a system architecture for the cubes system is created. Different modules and services were identified to help running apps and support system management and app development.
For better remote management all systems are based on a linux distribution (we chose Fedora Workstation 38). The cubes are running with a GUI, the server is headless.
As the possibility to run certain apps on the cubes this apps are defined in more detail. An app for the cube system can be distributed on the server and the cube:
- CubeApp (cubeside): this is the main interactive app part running on the cubes to be driven and experienced by users. From media presentation to little games to fullscale multiplayer games everything the cube hardware offers is possible. This app receives processed sensor events from the cube hardware and can connect to a central communication broker to communicate with other cubes or the server.
- GamemasterApp (serverside) [Optional]: the server adds the possibility to run web apps which can control the app state and the app flow on the cubes (for which it also connects to the central communication broker). It could just be a passive controller for start/stop or app phases, add additional features like timers etc. Also it can be a more complex app which can interact with or manipulate the cubes, If cube neighbourhood is needed (a.k.a which cube is docked to another cubes side ?) this app can gather the basic sensor data of connected cubes and validate this to an app specific truth, e.g. you want to solve a puzzle you have to know which Cube is Which Tile and how they are interconnected.
Each app can use the basic functionality of the system (Sensor Events etc.) but can also define their own actions and interfaces to communicate. The apps are stored together with an app configuration on the server which handles the distribution. For indicating that an app is running the app should send an app-state at its start and stop. The app-state could also be used to provide data to the gamemaster app.
System to system communication (Cube to Cube, Cube to and from server ) is handled by MQTT. Therefore a broker service is running on the server. Every module which have to communicate over MQTT is using a MQTT client.
The cube-internal communication between the CubeApp and a backendservice providing sensor events is done via ZeroMQ.
All data (with few exceptions) is transferred by JSON. For the basic data models we have designed some interfaces which have to be adapted in the modules. The interfaces are described in more detail here.
For some basic functionality needed each system (cubes & server) has installed some basic tools, apllications and services..
- Docker: For running and deploying services and apps to the systems we are using the docker containerization runtime together with Docker compose.
- Syncthing: To easily deploy new apps or update services Syncthing is used. Apps can be deployed to the server after development and then deployed to the cubes. All cube services are built on the server and then can be deployed to the cubes.
- Cockpit: As the server & the cubes don't have mouse and keyboard attached we use Cockpit as a system management and observation tool. It offers a terminal, system overview etc. The cockpit running on the server can connect to the cockpit on the cubes and save ssh keys so there is an easy access to all systems.
- Git: For developing and later espescially app development Git is a must-have.
- Python: All the backend services written for this project are using python.
A cube is the main interactive device in the system. the main purpose is running apps for the user. To make this work and to make it eays to deploy and maintain the cube some services have to run (in addition to the common services mentioned above) and some setup is needed:
-
Backend Service (Python): This autostart service is always running in the background . It establishes a MQTT connection and sends a cube state datagram continuously so the server knows the cube is online. Then it gathers data from the sensorboard connected by USB. The data is processed into some Events and values mentioned at the beginning before sending it to the CubeApp by ZeroMQ.
This service runs as a docker compose service and can easily be updated by restarting pulling an image built on the server.
-
Sensorboard (C++): The sensorboard attached to the USB port processes data from IMU, OpticalFlow and 4 NFC sensors for neighbourhood detection and sends it out to the USB by JSON.
To update the board there is also a docker composition which deploys the new code by just pulling a new image from the server and running the composition.
-
Syncthing: As mentioned above each cube is running a syncthing daemon in the background. A fodler from the filesystem is setup as a sync directory for cube apps. If the server-side app directory gets updated it is automatically synchronized with the cubes.
-
CubeApp (Unity): The cube apps resides in a dedicated directory which is synced with the server. Currently the app has to be an executable which can be started from the server. the app should implement the required interfaces together with MQTT and ZeroMQ connectivity. Our preferred app development framework is Unity for which we have a base package with the basic funcitonality so only app-specific additions have to be made.
The server provides centralized functionality for the cubes eco system like app & system management and communication enabling by using some peripheral services together with a developed cube server module.
-
Backend Service (Python):
The server hosts a backend service (also auto-start) which can be subdivided into multiple modules and functionality:
-
Communication: like the cube baclkend service the server service provides the MQTT client which establishes a connection to the broker
-
Cube control: For controlling the cubes there defined some commands to be run on the cubes by SSH:
- set volume
- start/stop app
- update backend service
- update sensor board
- restart cube
-
App management: For providing the stored apps to the system control dashboard the service regularily scans for apps (which have to be properly configured by a configuration file) in a dedicated directory and sends them out to MQTT.
-
Webserver: For provision of the webfrontends for system & app control as well as the gamemaster apps a webserver is configured. This webserver also can be used for REST-APIs or app-specific data (media & configs placed into the apps webroot resource folder).
Currently the neighbourhood configuration (NFC-Tag-Ids mapped to a Cube and his side) is accessible by REST.
-
Tracking data provisioning: if an optional positional tracking system is used the server can process the data and translate it into messages used inside the cube system
-
Neighborhood gathering: Also the neighbourhood from the single cubes is gathered and synchronized on the server and sent out via MQTT to be verified by a gamemaster app.
-
-
System & App frontend (Vue.js): the server runs a web application which shhows the usable apps with an interface to start or stop them. Also there is a cube view which shows all cubes with their state together with some controls to trigger the system actions on the backend. Another view just gives access to the peripheral services running on the server for deeper insights.
-
App registry: The app registry is just a filesystem directory on the server where apps are stored with their config file and the seperated apps (cubeApp & gamemasterApp).
-
MQTT broker: The central communication broker instance
-
Docker registry: a registry to store docker images of the cubes services
-
Build & Update helper: There are some setups for easy updating the cube services (backend & sensorboard) and building Unity-Apps by script on the server:
- Cube Upating: The source code of the sensorboard and the cube backend service is stored in a directory and can be managed by git. They are wrapped by a docker compose builder which creates new images which can be stored in the private docker regsitry,
- CubeApp building: The server has installed a licensed Unity versionto build the apps if provided by source in their folders. Therefore a build script is lying in the apps folder which can be copied to the app folder. If executed the headless build runs. if okay a new folder is created with the build. otherwise you have to check the logs and fix errors.
- GamemasterApp: To build a source-provided gamemaster app (based on ReactJS) you can use the npm package manager to install all dependencies and then run the build. A build folder is created which has to be synced with the config file.
-
Syncthing: The CubeApps stored onthe server are setup in a synced folder which can be downsynced by the cubes (readonly). Also the cube service descriptions (docker-compose-files) are added as a sync folder. This folder is synced on the cubes after first setup and on changes.
-
KVM/libvirt with UI: For hosting virtual machines like tracking systems a KVM/libvirt system is installed with a web UI.
-
Ubitrack: This tracking system comes as a VirtualBox VM which we exported as QEMU to be run inside the KVM system. If installed and configured correctly it starts an own MQTT broker from which the backend service should get the necessary positioning data.