The App essentially has 6 major components in 3 layers of software namely the sensor driving layer, In-vehicle application layer and the cloud services layer. The datapath across these layers follows a bottom to Top fashion. The Sensor driving layer implements the accelerometer service that collects and identifies sharp sudden peaks in 2 Dimensions. The peak detector software algorithm is calaibrated propery with real testing on road. It also implements the Camera Service with OpenCV video capture to process the objects detected in the frame and accurately estimate the distance of the objects. The algorithm used is a deep neural network based YoLoNet v4. It temporarily computes the speed of the vehicle based on the distance travlelled over a timespan but is extendable to integrate with the digital speedometer of the car. Integration of Accelerometer, Camera to detect a collision improves the accuracy of the collision detection algorithm as it's authenticated with multiple sensors.
The invehicle app suite orchestrates both the cloud and the sensor layer. Also, The app gives the necessary additional details to the cloud such as contact names, user music preferences etc. The app implements a scheduler, volume control button handlers used to increase/decrease volume. Media player in the form of Youtube Picture-in-picture. The entire app is built using React Native.
Coming to the cloud services, each service is implemented via an API. We made use of Django Framework. Firstly, the location and cognitive load is retrieved using the Google Maps API. Also, the same data is utilized in the computation of speed. Now, another major highlight of the project is location aware music prediction algorithm. This algorithm utilizes a machine learning algorithm of collaborative filtering making a matrix of artist x genre mapping using alternative least squares approach to assign a similarity score to the tracks the user might be interested in. We made use of .dat files from the LastFM dataset to sample from a random user past tracks / preferences. Location based filtering is adopted on the top 5 artists results generated using the Spotify Million Song DataSet. Now, A Youtube Music API is utilized to pass the artist and play the top 5 most played songs of that particular artist. The ML model generates unique set of recommendations each time it's trained on a sample. Next, the sound at which this music has to be played is also autonoumously determined and personalized by the software based on the cognitive load context. We implemented a moving average calculator with a configurable window size of 5 which is essentially a record of user's last 5 sound choices. Whenever user increases or decreases volume, the mechanism learns and adapts to the context. therafter, whenever a change is context is detected, the personalized sound is set each time. Voicemail service is another feature which the project provides to manage the driver's effort in answering/rejecting the calls with a personalized voicemail with a time after which the driver might be free. The cloud also implements a fuzzy logic mechanism to assign a driving score to the driver taking into account different contexts of Cognitive workload, Jerks determined by the peak detector in 2 dimensions and Speed as determined by the speedometer. All this data is streamed to the cloud every 3 seconds. Driving Score is computed using a fuzzy logic system, an example rule goes like this "In a Low Cognitive Workload, and In a High Speed case, If the Driver Brakes or changes highway lines rapidly thereby rising the accelerometer peaks" then the driving score points are reduced. We made use of this score to do a fuel budget calculation for the trip.
When a car moves left to right, there will be a change in the X-axis and when the car brakes, there will be a change in the Z-axis. There is a peak detector algorithm tuned to recognize these based on a threshold. There's another threshold we came up with fully calaibrated to detect the peaks that implies a sudden change in speed/collision. Whenever the accelerometer detects a collision, it calls a Django API. Now, There's a camera based Car Detection and distance estimation functionality built on the server. The Camera service is implemented by OpenCV Video Capture and Utilizes YoLoNet v4 Deep NN that's trained with images of cars. So, Whenever the car is driving and the algorithm detects a car as close as 30 inches (Fixed this as the threshold as the vehicle may be lost in sight), then it raises a flag. The in-vehicle app suite implements a scheduler and polls the cloud service every 3 seconds to check for a second confirmation on the collision which the camera service flags if detected. On the event of this multi factor authentication on the collision, the in-vehicle app suite sends voicemails to emergency services and emergency contacts.
Location-aware music prediction algorithm is implemented on the cloud. The system utilizes collaborative filtering, an ML algorithm which does artists-genres multiplication matrix and employs alternating least squares approach to filter the best match results based on the user's past listening/preferences. The algorithm is coded in python and hosted on a Django server. LastFM dataset is used to sample the data and train the model. Spotify million dataset is used as a test set to predict the songs with a similarity score. Location based filtering is done to retrieve the songs based on location. the artists data is made available to the client via Django API. The in-vehicle app suite calls the server whenever there's a change in location context. Upon receivivng the top 5 artists, it calls youtube music api to play the top 5 most played songs of the respective artist.
To support the driver managing the sound level, Volume personalization is done at the client in the in-vehicle app level. Volume UP and DOWN buttons are implemented in the UI. On increase/decrease events, the system personalizes the sound to the context. The algorithm records the user sound changing actions to adapt to his behavior. We implemented a moving average calculator with a configurable window of size 5 in JavaScript React-Native.
When the car undergoes lateral movement, a shift in the X-axis occurs, and when it brakes, a change in the Z-axis is registered. A finely-tuned peak detector algorithm, governed by a specific threshold, identifies these changes. Another calibrated threshold is established to detect peaks indicative of abrupt speed changes or collisions. Upon detecting a collision, the accelerometer triggers a call to a Django API. The server incorporates a camera-based Car Detection and distance estimation feature, leveraging OpenCV Video Capture and the YoLoNet v4 Deep Neural Network trained on car images. If the algorithm detects a vehicle within 30 inches, a flag is raised. The in-vehicle app suite, equipped with a scheduler, regularly polls the cloud service every 3 seconds for a second confirmation on the collision. In the case of multi-factor authentication, the app suite sends voicemails to emergency services and contacts.
The cloud hosts a location-aware music prediction algorithm utilizing collaborative filtering. This ML algorithm involves an artists-genres multiplication matrix and employs an alternating least squares approach to filter the best matches based on the user's past preferences. Written in Python and hosted on a Django server, the algorithm is trained on the LastFM dataset. The Spotify Million Song dataset serves as a test set for predicting songs with similarity scores. Location-based filtering retrieves songs based on the user's location, and the Django API exposes artist data to the in-vehicle app suite. The app suite communicates with the server when there's a change in location context, fetching the top 5 artists and using the YouTube Music API to play their top 5 songs.
To enhance the driver's sound management experience, volume personalization is performed at the in-vehicle app level. UI elements, such as Volume UP and DOWN buttons, facilitate user interaction. On volume adjustment events, the system tailors the sound to the context, adapting to the user's behavior by recording sound-changing actions. A JavaScript React-Native implementation includes a moving average calculator with a configurable window size of 5.