Support for multi-channel audio streaming

[leo150]

PR

• Feature: add multi channel audio support

Description & motivation

This change is providing an audio layout for the AVAudioFormat when number for channels of the input format is more than two. AVAudioFormat recommends using init(streamDescription:channelLayout:):

If the AudioStreamBasicDescription specifies more than two channels, this method fails and returns nil.
Instead, use the init(streamDescription:channelLayout:)

This PR adds a method to create a layout for non mono/stereo audio formats. It also limits maximum number of output audio channels to two in AudioCodecSettings. Lastly, it adds a method that creates a channel mapping and provides it to the audio converter.

I tested this change with mono, stereo and 4 channel audio. Mono and stereo fall back to the previous behavior. It's possible to use channel layout as well. However, I decided to fall back to the previous implementation instead this one:

static func makeChannelLayout(_ numberOfChannels: UInt32) -> AVAudioChannelLayout? {
    switch numberOfChannels {
    case 0: return nil
    case 1: return AVAudioChannelLayout(layoutTag: kAudioChannelLayoutTag_Mono)
    case 2: return AVAudioChannelLayout(layoutTag: kAudioChannelLayoutTag_Stereo)
    default: return AVAudioChannelLayout(layoutTag: kAudioChannelLayoutTag_DiscreteInOrder | numberOfChannels)
    }
}

What didn't work at all is to use custom channel descriptions:

Code

static func makeChannelLayout(_ numberOfChannels: UInt32) -> AVAudioChannelLayout? {
    let channelDescriptionsPtr = UnsafeMutablePointer<AudioChannelDescription>.allocate(capacity: Int(numberOfChannels))
    
    // code for numberOfChannels >= 4
    for index in 0 ..< numberOfChannels {
        channelDescriptionsPtr[Int(index)] = AudioChannelDescription(
                mChannelLabel: AudioChannelLabel(index),
                mChannelFlags: [],
                mCoordinates: (0, 0, 0)
        )
    }
    
    var channelLayout = AudioChannelLayout(
            mChannelLayoutTag: kAudioChannelLayoutTag_UseChannelDescriptions,
            mChannelBitmap: [],
            mNumberChannelDescriptions: numberOfChannels,
            mChannelDescriptions: channelDescriptionsPtr.pointee
    )
    
    return AVAudioChannelLayout(layout: &channelLayout)
}

Every call of AVAudioChannelLayout(layout: &channelLayout) generates AVAudioChannelLayout instance with random number of channels. I wasn't able to find any combination of mChannelLabel to create a stable result. Ring buffer and converter end up having different audio formats which leads to a crash: required condition is false: [impl->_inputBufferReceived.format isEqual: impl->_inputFormat].

Examples of random channel layouts

All instances of AVAudioChannelLayout was provided with the same AudioChannelLayout. AVAudioChannelLayout attempts to make a conversion as states in the doc. Somehow it creates random layout every time:
Example 1:

"tag: Unknown, bitmap: AudioChannelBitmap(rawValue: 36915), channels: 51, channelLabels: ["Unused 0", "Unused 0", "Unused 0", "Unused 0", "Unused 0", "Unused 0", "Unused 0", "Left 1", "Unknown 4294967295", "Unknown 582390464", "Unused 0", "Unused 0", "Unused 0", "Unused 0", "Left 1", "Unknown 2147783024", ..."

Example 2:

"tag: Unknown, bitmap: AudioChannelBitmap(rawValue: 36915), channels: 143, channelLabels: ["Unused 0", "Unused 0", "Unused 0", "Unused 0", "Unused 0", "Unknown 1400468065", "Unknown 1936942419", "Unknown 1649632357", "Unknown 1600279396", "Unknown 582482080", "Unused 0", "Unused 0", "Unused 0", "Unused 0", "Unknown 36915", "Unused 0", "Unused 0", "Unused 0", "Unused 0", "LeftSurroundDirect 10", "Unused 0", "Unused 0", "Unused 0", "Unused 0", "Unknown 1852795252", "Unknown 1633971829", "Unknown 1953391972", "Unknown 1918985326", "Unused 0", "Unknown 1098085743", "Unknown 1835099506", "Unknown 1769234787", "Unknown 1600939364", "Unknown 3935328888", "Right 2", "Unused 0", "Unused 0", "Unused 0", ..."

Example 3:

"tag: Unknown, bitmap: AudioChannelBitmap(rawValue: 36915), channels: 36, channelLabels: ["Unused 0", "Unused 0", "Unused 0", "Unused 0", "Unused 0", "Unused 0", "Unused 0", "Unused 0", "Unused 0", "Unknown 3935305977", "Unknown 792359792", "Unknown 1734633847", "Unknown 1969434488", "Unknown 112", "Unknown 4294967295", "Unknown 4294967295", "Unused 0", "Unused 0", "Unused 0", "Right 2", "Right 2", "Right 2", "Unused 0", "Unused 0", "Unused 0", "Unknown 2147849040", "Unused 0", "Unused 0", "Unused 0", "Unknown 106152448", "Unused 0", "Unused 0", "Unknown 4294967295", "Unknown 4294967295", "Unused 0", "Unused 0"]"

More discussion in this issue: #1262

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)

[shogo4405]

I'm currently in the process of ordering equipment that can output audio in 4 channels. We will merge it after testing. Thank you very much.

[shogo4405]

😅

[leo150]

Thank you for taking a look at this! I've changed tab indentations to spaces and removed unnecessary change of makeAudioFormat.

[shogo4405]

I was able to test with 4ch/6ch (32bit) using macOS loopback (https://rogueamoeba.com/loopback/). While I haven't been able to test multi-channel support on iOS yet, I believe it will work and will proceed with the merge. Thank you for your contribution.