My goal is to accurately provide the DJ/manager with real-time data about energy levels in the crowd, where, in the frame of reference, the crowd is more "hype", and to sync different processes based on crowd engagement and preference. One of these features includes accessing attendees' Spotify data with their permission to extract crowd favorite genres, which are ranked in real-time by popularity. Another important feature I chose to integrate was synchronizing lights in a venue to match the energy levels of the crowd.
I built Vibe Check by starting from the core of this project, which was optical flow tracking with OpenCV. I ran this and tested its accuracy in low-light settings to assess its feasibility with my vision. Then, I worked to extract the intensity of movements from the Farneback optical flow method so that I could vectorize each pixel in successive frames and measure movement. After this, I added weights so that I could calculate a real score for hype and energy levels. Then, I split the frame into a grid and figured out where in the grid energy movement was coming from. This allowed me to create a heatmap of energy in the frame.
To display this, I built a React user interface and used WebSocket to host it seamlessly on my local machine. I then connected this data to an InfluxDB database in which I could store energy levels by timestamp and per frame for later use. From here, I brainstormed additional features to add. For example, I thought to add a feature that allows attendees to scan a QR code and then give permission to access their Spotify listening data in exchange for a voucher of sorts. I have shown this as a proof of concept on the Vibe Check dashboard by simulating a demo stream of check-ins and popular genre rankings for the DJ to adapt to. Another feature I thought to demonstrate was integrating the energy level with venue lighting systems so that they can adapt accordingly.