Making interior decoration fun!
曾元 B07901163
林佩潁 B07901102
邵家澤 B07901081
Doing interior decoration is not easy. Traditional interior design is done by meticulously creating a 3D model of your house, then spending agonizing hours in front of your computer, wondering where to put what. We want to make the process straightforward and interactive, so that users can walk around their new house and with a click, place furniture anywhere they like.
Slides for our final presentation can be found here.
recall.mp4
We use STM32 board and Raspberry Pis to create an interactive way for interior design. By setting up RSSI network, we can do the triangulation and find the user's location. The user can press the button on the STM32 board to trigger IRQ function for setting furniture. The STM32 board also supports accelerometer and magnetometer for motion and heading detection, which allows user to switch object and change facing direction. The user can walk around the room and place the furniture at anywhere he/she wants. We demonstrate our results with the Minecraft game engine.
Compared with outdoor localization, indoor localization requires much higher precision to be useful. In our case, we would like to differentiate between objects within 2-3 meters of each other, whichs rules out GPS or other outdoor localization methods. After surveying several papers, we decided to try triangulation, using multiple RSSI measurements of a Bluetooth signal. Our system consists of a STM32 board that advertises its signal, and 4 Raspberry Pis that detect the RSSI.
The received signal strength indication (RSSI) method is one of the common choices for interior positioning. The RSSI value represents the power of a received radio signal. The higher the RSSI value, the higher the signal strength. Moreover, since no additional sensors are required to measure RSSI values. It becomes our first choice.
RSSI distance method requires the received signal strength from the Bluetooth anchor points. The signal strength would decay as the distance between transmitter and receiver increase. Thus, we can build a decaying model and use the RSSI value to estimates the user’s distance respect to the anchor point.
We use the mainstream logarithmic distance path-loss model to calculate the distance from the RSSI value. The mathematical decaying model is expressed as below. d is the distance between the transmitter and the receiver, and n is a decaying factor related to the specific wireless transmission environment. The initialization process is essential to find the decaying factor n.
Our model assumes that the RSSI value is only dependent on the distance between the two devices. However, in reality, RSSI values are significantly influenced by the environment and noise caused by multi-path fading. There exist multiple reflection paths for the signal to transmit from the transmitter to the receiver, which may cause some interference so the actual received strength might be quite different from the ideal model. As a result, the variation of the received RSSI are unstable even in a well-controlled indoor scenario. Thus, some filter and post-processing is needed.
We adopt Kalman filter on both received RSSI value and the calculated distance to remove some noise and eliminate the large variation. The Kalman filter is a state estimator that makes an estimate based on noisy measurements. The key is that it takes the history values as well as the uncertainty of measurements into account.
After we obtain the distance from each AP with known location, we can do the triangulation to find the coordinate of the user. We use scipy.optimize library to find the optimized coordinate and minimize the error.
We use the MQTT messaging protocol for communications between Rpi and the server. MQTT is a publish-subscribe network protocol that transports messages between devices, which is a great choice for communication between different IOT sensors.
Rpi act as MQTT publisher and the server act as MQTT subscriber. The benefit of using MQTT is that the message is organized in a hierarchy of topics. We can specify the topic for the publisher and subscriber. In other words, it allows us to differentiate the message from different Rpi simply by the topic. The subscriber can subscribe to multiple topic at the same time, and the publisher can publish their message no matter the subscriber exist or not. This allows a great flexibility for our setting.
In conclusion, we set four Rpi at each corner of the room. Each AP would publish time stamp and the calculated distance to the server every four seconds. When the server receives four distances with the same time stamp, it would do triangulation and find the coordinate.
In other words, the syncronization among four Rpi are achieved by only send the message when the time stamp is the multiples of 4. So that the server can check the time stamp to find the corresponding distance pair.
Moreover, the average window of three values is used for post-processing on calculated result.
path_follow.mp4
We set another STM32 as an remote controller to signal the PC end the detections on the board. They would become the APIs for the 3D Modelling. Making use of the MBed wifi example and the python file as a listener on PC, the STM32 and the PC are connected.
We then read the sensors from the STM32 and send the instructions to the PC. The accelerometer is set to detect the movement of hands to signal the PC either it is pointing up or down, or flipping right or left. Since the hand movement is supposed to be big enough so that it can be executed, a simple threshold is set to aviod noises.
The compass function is formulated by the data combination of both accelerometer and magnetometer. Once we acquire the six axis of the data, the heading is obtained by the tilt compensation algorithm. Accelerometers sense the overall acceleration (gravity) ,meanwhile the magnetometer gives the direction of the magnetic north. Therefore with the algorithm, the 6 axis can be transformed the (row, pitch, yaw) directions, where the yaw is the heading we need for.
Embedded.Final.STM32.and.Minecraft.Function.Demo.mp4
We originally planned on using SketchUp or other 3D modelling software commonly used for interior planning to place actual pieces of furniture in a 3D room model, but none were freely available or had more inconvenient APIs. We then decided to switch to Minecraft, a popular sandbox game where players explore a 3D world made entirely out of blocks. Minecraft is supported on many devices, and also has a Python API originally made for its Raspberry Pi version, but later extended to all devices capable of hosting a Minecraft server.
We use a modified version of this API1 that allows us to place blocks and move the player according to how the movements and actions of the user in real life. As shown in the video above, we can control the rotation(left-right), pitch(up-down) based on movement detected by the accelerometer and magnetometer of the STM32 board.
We set four pillars of blocks to mark the position of the four Raspberry Pis in-game . As shown in the two videos below, we see the user moving from the gray, to the white, then to the blue pillar in the lower video, which corresponds to the respective Raspberry Pis in the upper video.
walking_demo.mp4
minecraft_demo.mp4
TBA
-
Synchronization:
Currently, our Raspberry Pi devices send data only when the time stamp is the multiples of 4. So the server can check the time stamp to synchronize those four devices. When the server receives four distances with the same time stamp, it would do triangulation and find the coordinate.
However, the process has resulted in the precision limitation since the time slot between two calculated positions would be at least 4 seconds. Not to mention that we may need to wait additional 4 seconds if one of the devices fail to scan the RSSI value. Since the update rate is limited, the response time of the indoor positioning would be slow, thus limit in precision scale.
As a result, we would like to find a different approach for the synchronization, such as using distributed key-value store instead of sending distance on MQTT. We may also skip the Kalman filtering on raw RSSI data, since now we can only get approximately three or four values in three seconds scanning time. The current filtering effect is not obvious because of inadequate amount of data. Thus, we may trade off the filtering precision for more data points.
Moreover, the optimizing algorithm should be rewritten, since now we only optimize for the coordinate when all four distances are received. So when only three of the devices are found, we would not calculate the coordinate based on those distance pair. But it would waste a lot of time to wait for all four devices working properly. Thus, the algorithm should be able to handle both cases. We could also use a better RSSI scanner like ibeacon.
-
Extending our system to function on actual 3D modelling software.
[1] Chai, Song & An, Renbo & Du, Zhengzhong. (2016). "An Indoor Positioning Algorithm using Bluetooth Low Energy RSSI," 10.2991/amsee-16.2016.72.
[2] Xiuyan Zhu, Yuan Feng. (2013) "RSSI-based Algorithm for Indoor Localization, Communications and Network," Vol.5 No.2B
[3] Kalman Filter: https://www.kalmanfilter.net/kalman1d.html
Footnotes
-
Minecraft Python API Usage & Installation: https://github.com/stoneskin/python-minecraft ↩