2022 Spring Hybrid Teaching for Music and Machine Learning
Jakob Høydal, Joachim Poutaraud, Hugh Alexander von Arnim, Sofia González Muñoz, Oliver Getz Rodahl, Arvid Andreas Falch, Kristian Wentzel, Joseph Clemente
This wiki-page is a run-through of a physical-virtual teaching setup without using zoom for audio.
We did two trials with two different setups based on the concept of extending the physical space:
- Setup A) Stereo-spatialisation
- Setup B) Multi-spatialisation
Setup A) Stereo-spatialisation was based on extension through a central screen/ speaker array. This is a further development of setup B), stripped down to an audio stereo-projection at both sides, from a stereo pair.
Figure 1 - Stero-Spatialisation
Setup B) Multi-spatialisation was an attempt to extend space without focus on a central screen/speaker array. This comprised four microphones on each side, projecting to four speakers on both sides. See figure 2 for the initial layout, and section 4 for microphone-to-speaker projection.
Figure 2 - Multi-Spatialisation
This wiki-page contains information about Setup A) Stereo-spatialisation as that was the most successful setup, but it’s also possible to further extend on the things we learned from Setup B) Multi-spatialisation.
Each desk of two persons was mic'd up using one AKG C414 microphone in cardioid pattern. In addition one AKG G30 was placed on the desk of the lecturer. We also tried an alternative approach with each desk miced up with a stereo pair of microphones, either a pair of AKG C414 or pairs of T Bone Em 9600 (the shotguns) in cardioid pattern. We found that the T Bone Em 9600 sounded a bit muffled and was less preferable than the AKG C414 for high quality sound. The stereo pairs did not really improve the experience and we therefore opted to go with one microphone per table for simplicity's sake.
The microphones were routed through the mixer and then into either LOLA (first attempt) or Jacktrip (second attempt). The sound from the other location was broadcasted using a stereo pair of Genelecs. For the Portal this setup was pretty straight forward using the stagebox and Midas mixer.
This is a brief explanation of the audio routing in the Dungeon. We had to make some specific routing choices worth mentioning, since this room has a Mackie 32-8 analogue mixing console not as flexible as the digital Midas M32 in the Portal.
The audio routing in the Dungeon was a bit cumbersome, as the analogue mixer we currently use has its limitations (and flaws). Some time was spent in troubleshooting noisy signals by trying out different cables, connectors, channel strips and speakers. In the end we found that the main left output was noisy and affected the audio signal. This is the reason for us using bus sends 7-8 connected to the left/right speakers as our main output for this session. We would’ve used the regular Main L/R outputs if they worked properly.
The AKG G30 (Goose-Mic) was routed to both bus 3 and 4 to center the lecturer in the middle of the stereo field. In the diagram, the outputs of the mixer show the actual bus output channels suggested for a working setup with our current hardware. All soundcard I/O channels and mixer input channels are otherwise chosen as deemed appropriate for the occasion.
In our setup, the 4 channels sent through the network were divided into two stereo groups: Microphones L/R (channels 1-2) and AUX L/R (channels 3-4).
When doing hybrid teaching, two screens are often better than one:
- a content screen
- a participants’ screen
This strategy allows the teacher to see their students and not feel like they are talking into a wall while displaying content on one of the screens, and students will feel like part of a classroom environment. When employing a setup with more than 1 screen, take care to reduce physical stress and encourage interaction.
For a complete guide to video communications setup in the Portal and Video room, see MCT Portal Documentation Setup & Routing. This setup is tested with a pair of PTZ cameras.
To make communications feel conversational and natural, for each location, aim a camera at the students and teacher from the front and place it below. This way, they will be facing each other through the video conferencing tool of your choice. This strategy will also limit any unnatural twisting and turning of the neck when communicating or retrieving information from
A projector can be used in place of one (or more) of the monitors. Depending on the projection surface (the wall) and the quality of the projector, some distortion or unclarity should be expected. It is recommended that you use a projector for less important information, such as to display other participants, or find a better, portable surface. Slides, code, visual examples, and similar content should be displayed on a screen.
There are currently two options for using a projector:
- Ceiling Projectors (There is one in the Video Room and one in the Portal).
- Worse quality, quicker setup.
- LoLa Portable Rack Projector (found in the drawer of a portable LoLa rack).
- Better quality, but not connected by default.
If your LoLa Portable Rack Projector is blinding the camera facing the participants, place the projector below or next to the camera and tilt it to taste.
- Screens perpendicular to the participants (fig. 2). This causes too much physical stress.
Our first attempt at tele-conferencing saw us using the low latency audio visual streaming system LoLa as we thought it could be easier to send all AV-signals in a singular software. However, because it was mandatory to share screens for the MCT4052 course, we finally agreed on splitting audio and video transmissions in two, using Zoom for video network connection and screen sharing and JackTrip to enhance the quality of our audio network connection.
We established a network connection with JackTrip using the following steps:
-
Enabling the public firewall and turning off the other ones (private and domain) on the LOLA computers
-
Checking network latency between the IP addresses of the Portal and the Videoroom doing a round-trip time (RTT) with the
Test-NetConnectioncommand line.
- Launching the Jack Audio Connection Toolkit (QjackCtl) with the following setup:
- Sample rate: 44100 Hz
- Frame/Period: 128
- Creating a
jacktripenvironment variable for the directory of thejacktrip.exefile location and using the Windows PowerShell to launch JackTrip in order to create a server connection.
Note: we could avoid using QjackCtl using the following command line:
jacktrip -s --rtaudio --srate 44100 --bufsize 128
However, using QjackCtl is definitely better for routing audio connections graphically.
We designed a p2p connection with 4 channels (i.e. two microphones and one stereo minijack) as it appeared appropriate to make a simple two-way connection to one jacktrip client with a jacktrip server in full mix mode (p4). This last parameter allowed Joseph to hear the full mix including himself.
- Joachim - Server:
jacktrip -s -p4 -n4 - Joseph - Client:
jacktrip -c 129.240.238.20 -p4 -n4
One of the main issues we were having in the portal was having too much feedback, but we were able to help the situation out by using the hub patch model number 2 instead of the default hub patch model. That way, everybody could hear everybody else, but not themselves.
- Joachim - Server:
jacktrip -s -p2 -n4 - Joseph - Client:
jacktrip -c 129.240.238.20 -p2 -n4
Moreover, I could route the 4 channels sended by Joseph to the selected outputs of the RME Fireface Interface using the QjackCtl Graph option. This allowed Kristian to receive the signals in order to mix and broadcast them to the main speakers of the Videoroom.
Finally, we received a connection from Peer! validation in JackTrip indicating that the network connection was enabled. We could therefore facilitate a Physical-Virtual Teaching sending 4 audio channels through JackTrip and 2 video streams using Zoom to share screens and display the camera connection.
Using JackTrip for audio, we ended up with an easy and successful setup with outstanding quality audio that, despite some small issues such as the delay between the audio and the video in each room, ended up feeling remarkably close to having everyone together in the same room.
As mentioned in the introduction, setup B) multi-spatialisation didn’t work. This was mainly because the position of the video-monitor and presentation-monitor were different. The participants had to repeatedly turn their heads to look at either the presenter or the presentation, which had a negative effect on the fluidity of the workshop.
A diagram of the concept for multi-spatialisation can be found in the attachment below. Microphones are spread out and projected in a four-speaker array behind the video-monitor.
It could be useful to further experiment with a four-microphones-four-speakers-projection. In our testing, we found that it helped a bit with the spatialisation of each participant. But since the presenter was the main focus and talked most of the time, over the course of a four-hour lesson this became a bit annoying for the participants on the other side.