2023 Mid Semester Networked Music Performance in Salen
Mid Semester Networked Music Performance with Edvard Munch Vgs Students, which was announced as "Playing Jazz Over Network" was performed on 7th of March in Salen and RITMO.
As a main software solution to establish the video and audio connection in two locations, we used Lola, and set the Lola connection between two NPMs in Salen and RITMO. The sampling rate was 48000 HZ while the buffer size was 64.
In Salen we were responsible for these instruments
- Acoustic Drums
- Grand Piano
- 3 Vocals
All audio was handled by the Midas M32 mixer present in Salen before sending it over the network to RITMO through a LoLa Kit Audio technicians on both sides were talking with each other constantly to find a way to get a balanced mix on both sides. The solution from the RITMO side was to send every instrument over separately as mono channels. This worked very well and made the job of balancing the outgoing audio in salen much easier.
In salen we had 8 separate microphones where 4 of them were stereo-pairs used for drum overheads and piano. Therefore we decided to balance everything in the mixer before sending everything to a stereo bus and then to LoLa, this took quite a bit of time to set up.
Once the musicians arrived we figured out we needed another monitor for the vocalists since we weren't sure how many there would be on stage at once. This led to some issues with the mixer and stagebox and no audio could be sent to the second monitor. We fixed this by connecting the first monitor to the second one. This would ensure all singers had adequate, although not ideal, monitoring. The other performers on stage had no issues with the monitoring situation. We wanted to use monitors and not headsets since we thought it would look better for the audience.
For capturing performers in Salen, we used Ximea MQ013CG camera which was connected to LOLA computer. We positioned the camera in the middle of the room so that all performers can be centralised in the frame. We had some problems with the color palette and had to customise color settings through Lola.
For displaying the performers in RITMO, we used two monitors and one projector all connected to the Lola machine by HDMI.
The video from the performers in RITMO was streamed to one of the monitors and to the projector while another monitor was used for Lola settings and monitoring the video of Salen stage.
The monitor with RITMO performers was places on the stage at a point which drummer and singers can see and communicate with the other performers. The vide from the projector was positioned in the middle of the stage, on a black curtain, so that the audience can have a feeling of seeing all the performers together.
We deployed a simple loopback system we already worked with in class for measuring the system’s latency on both stage days. A click track was sent via USB interface to the Salen NMP kit which sent the signal to RITMO. There it was looped back to Salen and recorded in Ableton. Another loopback was set up in the USB interface to exclude adding latency through the same.
We measured a Round Trip Latency Time of 14ms on rehearsal day and 17ms on concert day which falls well below the required 25-30ms latency threshold for performability. We worked with a 64 samples buffer size and 48k sampling rate. As a consequence, there was no need to revert back to a latency-accepting approach.
The concert was streamed on YouTube using the Salen streaming setup, comprising two ceiling cameras, a control station, and a computer with OBS Studio streaming software. To ensure high-quality audio, we connected the mixer directly to the streaming setup. During the live event, we utilized two cameras interchangeably and carefully synced the audio and video through manual measurements on testing streams before going live, delaying the audio by 385 milliseconds. The recording is available on YouTube at this link.
- We could've spent more time figuring out a way of sending mono channels to RITMO although our time in salen sadly got cut short because of a meeting.
- We could think of a larger pool of monitors to use for setting up the stage correctly for a larger ensemble. Since we only had one high-speed monitor at our disposal we had to place it where the drummer was sitting, all the way on stage-left. This does not represent an ideal solution for the other musicians and could be solved by setting up another monitor or utilizing multiple smaller monitors that do not disrupt the stage’s visual integrity.
- To make a seamless integration between performers on network and performers on stage, it could be interesting to have identical lighting setups at both locations and send the DMX signal over the network so lighting could be identical for network and stage performers.
- Having a contingency plan in place for streaming audio and video would have been beneficial. Unfortunately, we experienced technical difficulties with the audio connection on the streaming computer just before the concert, which caused some challenges and bad audio in the beginning of the live stream.