Wiki

Concepts learned and explained

This section serves to explain some of the concepts behind the architectural choices made in the EcoHub project.

asyncio.Queue vs. queue.Queue

The controller uses a queue from the asyncio module, specifically designed to be used in async/await code. The get() method of a asyncio.Queue waits (await) until an item is available. This means that the controller is on standby until a packet becomes available. Using await pauses the coroutine (the async function) without blocking the entire program, but instead giving control back to the event loop (the other functions running in the program, i.e. the devices sending packets). When a packet becomes available, the event loop resumes the controller's consume() method from where it left off.

HOWEVER! Because the controller and the devices are running in the same main thread, ONLY one coroutine runs at a time. In other words, while the controller is processing a packet, the devices cannot send packets. This means the execution is concurrent ("fake parallelism") but not truly parallel. True parallelism would require multiple threads or processes.

Tasks and task groups

The controller's consume() coroutine (async method) is run as a task. The same goes for each of the connected devices' run() coroutines. These tasks are then grouped in a TaskGroup, which runs them together concurrently, managing them as a group. If each coroutine was run with await, there would be no concurrency, since the program would wait for a coroutine to finish (forever) before executing the next one.

Separate thread for storage worker

Since writing to a file is an input/output (I/O) operation, it by default blocks the thread (control is given to the operating system) the operation is running in until it is complete. Having the storage worker in the main thread along with the controller and devices sending packets constantly would block the entire program, thus blocking packet processing and packet sending. Therefore, by having the storage worker in a separate thread working separately, the main thread remains unaffected during the file write operation and can continue normal execution.

Storing object references in the controller

The devices execute commands through their execute_command() method. For the controller to communicate commands for devices to execute, the command must be passed to this method, which requires keeping a reference to the device object in order to call the method on.

This is a naive approach, as in real life, controllers would not communicate directly with device objects in memory. Instead, controllers and devices would be connected over a network and communicate through protocols such WebSockets or MQTT (most used in real-life IoT systems).

Improvements on current architecture

Using sockets (e.g. WebSockets) would eliminate the need for the controller to keep references to device objects in memory, as the controller and devices would communicate by binding and listening to a common port, and broadcasting messages (devices would broadcast status updates and controllers would broadcast commands for the device to execute.)

Another, more realistic and more fitting implementation is through using the MQTT (message queuing telemetry transport) protocol. In MQTT, each smart device is an MQTT client, that publishes messages on a specific topic (a topic dedicated to itself, e.g. 'home/bedroom/temperature') and subscribes to messages on this topic. An MQTT broker manages messages published, and ensures all clients subscribed to a topic receive the latest message. The controller is also an MQTT client that is subscribed to topics, to which devices publish messages to, but also publishes messages to topics (commands), such that devices subscribed to these topics execute the commands in the messages.