[bug] Stereo music is being upmixed in native OpenAL games, while on X-Fi it isn't

[Zoomer1988]

As the title says, when using openal-soft with prey in 5.1 speaker configuration, the music in game and in menu is being upmixed to all 5 speakers, most prominently abusing the center channel.

Prey (2006) has a native OpenAL support, so no redirection methods are needed. I have another system with X-Fi PCI card installed and when ct_oal is used, music stays in front left and front right only.

I've tried playing with openal-soft config utility, but none of the options alter the behavior: I've tried setting Stereo Encoding to basic, set channels to be explicitly 5.1 surround, also tried disabling effects on the "Render" tab (High Quality mode, Distance compensation, Near-Field Effects).

The Openal-soft version is 1.23.0, however it's not the first time I noticed this issue. I believe, it existed even before the EAX support was introduced.

Also, I think, this issue is not exclusive to Prey. I believe, way back I noticed it in some other OpenAL title. Maybe it was Unreal Tournament 3.

Yet, in other titles everything works properly. I tested it in Splinter Cell 1 with Alchemy and then DSOAL. The music plays as it should.

Am I missing something in config?

That's what I currently have in my global alsoft.ini:

[General]
sample-type=float32
channels=surround51
resampler=bsinc24
frequency=48000
stereo-mode=speakers

[decoder]
speaker-dist=1
hq-mode=true

There are no active system upmixers or enchancers. The OS is Windows 11. Currently openal-soft is installed as the only oal implementation. Generic software is deactivated.

[kcat]

OpenAL Soft uses virtualization for multi-channel (including stereo) sounds. This helps ensure the sound has similar positioning regardless of using speakers or HRTF, or whether it's a wide or narrow speaker configuration, though it does result in a slightly more diffuse response or "speaker bleed" (which is a side effect of the panning method used to help ensure consistent directional accuracy, and allow dynamic adjustments for the AL_EXT_STEREO_ANGLES and AL_SOFT_source_spatialize extensions).

The app can disable that behavior and have multi-channel sounds play directly on output speakers (with a compatible output mode) using the AL_SOFT_direct_channels[_remix] extension, though it should be done with a bit of care since direct channels with headphones won't sound good if it's not a binaural stereo sound made for headphones (and certainly not if it's surround sound).

[Zoomer1988]

Thank you for such a detailed write-up!

So, in other words, this is expected? And the hardware OpenAL doesn't do that with music presumably because the panning algorithm differs, am I correct?

Then, in other cases where the music wasn't affected (e.g. the aforementioned Splinter Cell 1) its source probably wasn't put in a 3d environment? Or DirectSound3D api maybe had a special flag for the music source to avoid panning. And that's why when using DS3D->OAL path, the music still isn't being upmixed/panned since the flag is preserved.

Alright, that clears it up. However, is there a possibility that there's a similar "flag" or property in OpenAL specs for the sound source that is not expected to be panned or bleeded into other speakers, that openal-soft misses somehow? Could it be in EAX extensions?

I imagine, they should've thought of such a path (i.e. AL_SOFT_direct_channels[_remix] alternative for specific sounds like music) because the upmixed music sounds really bad. Stereo separation is almost non-existent and like 80% of it is crammed into the center channel, which is most often not a full range speaker and is not built for music at all.

I can conduct more tests to find the instances in native OpenAL game titles with EAX support, where the music remains unprocessed on hardware OAL and it's the other way around in Openal-soft. If that's needed.

Edit: I just remembered another OpenAL EAX title which doesn't behave like that. It's The Lord of the Rings online. Earlier they advertised EAX support and indeed they had at least EAX reverb working. Then they started having lots of problems with it (X-Fi bsod driver bug and then later missing sounds with XFi Titanium due to X-RAM bug) and decided to switch to openal-soft by default sometime by 2020. Still, music isn't being upmixed in that title.

Though again, they might be not putting the music source into the 3d environment... Oh well

[kcat]

So, in other words, this is expected? And the hardware OpenAL doesn't do that with music presumably because the panning algorithm differs, am I correct?

Right, and hardware doesn't have the AL_EXT_STEREO_ANGLES and AL_SOFT_source_spatialize extensions for dynamically adjusting/moving multichannel sounds to deal with. It can selectively decide when to play multichannel buffers with or without virtualization since the app has no control over it (or just not provide certain formats when output can't handle it).

Then, in other cases where the music wasn't affected (e.g. the aforementioned Splinter Cell 1) its source probably wasn't put in a 3d environment? Or DirectSound3D api maybe had a special flag for the music source to avoid panning. And that's why when using DS3D->OAL path, the music still isn't being upmixed/panned since the flag is preserved.

DSOAL should behave the same as OpenAL Soft, since DSOAL uses OpenAL Soft to mix/render audio. ALchemy uses hardware OpenAL, so would behave the same as hardware.

However, is there a possibility that there's a similar "flag" or property in OpenAL specs for the sound source that is not expected to be panned or bleeded into other speakers, that openal-soft misses somehow?

Neither DSound3D or standard OpenAL have a property to indicate if a multichannel sound should be virtualized or not. Standard OpenAL only specifies that multichannel formats don't do 3D processing (the position, velocity, cone, etc, properties are ignored), but that's separate from channel virtualization (local panning of each buffer channel).

I imagine, they should've thought of such a path (i.e. AL_SOFT_direct_channels[_remix] alternative for specific sounds like music) because the upmixed music sounds really bad. Stereo separation is almost non-existent and like 80% of it is crammed into the center channel, which is most often not a full range speaker and is not built for music at all.

The standard OpenAL API was originally designed about 20 years ago to help Loki Software with porting apps that used DSound3D to Linux. Creative then jumped on board to support their hardware with it. Since stereo formats are the only standard multichannel formats, and surround sound devices would have matching surround sound extension formats, multichannel formats were likely expected to just be simple output since that's generally just what apps needed back then. But things got more advanced over time, so the implementation and feature-set needed to adapt.

5.1 output has unfortunately always been problematic as well, being a tug-of-war between directional accuracy vs speaker sharpness, and directional accuracy having its own issue due to the irregular speaker spacing. The sharper you allow sounds to be on a particular speaker, the less accurately positioned sounds will seem when between speakers. I've tried different things, with more stereo separation getting complaints of the center speaker not being used properly.

You can use a different decoder preset that excludes the center channel from being used with panned sounds, if it's not a full range speaker or you just prefer not to use it. OpenAL Soft comes with a presets/itu5.1-nocenter.ambdec decoder preset file, and you can use the alsoft-config.exe utility to select it for 5.1 output (under the Renderer tab, in the Decoder Configurations section, click Browse... next to the 5.1 Surround entry and pick that preset file, then click Apply). That may help stereo separation. You can also try the square.ambdec (for the front and surround speakers in a square configuration) or rectangle.ambdec (for the left and right speakers being closer to each other) presets, though they're more for quad instead of 5.1 setups.

It's The Lord of the Rings online. Earlier they advertised EAX support and indeed they had at least EAX reverb working. Then they started having lots of problems with it (X-Fi bsod driver bug and then later missing sounds with XFi Titanium due to X-RAM bug) and decided to switch to openal-soft by default sometime by 2020. Still, music isn't being upmixed in that title.

It's possible they're using one of the AL_SOFT_direct_channels[_remix] extensions since you say they switched to OpenAL Soft (an app doesn't actually have to "switch to" OpenAL Soft to use the extension, but it does need to be aware of and use it when available).

[Zoomer1988]

Again, thanks for such detailed answers!

when using DS3D->OAL path

DSOAL should behave the same as OpenAL Soft, since DSOAL uses OpenAL Soft to mix/render audio. ALchemy uses hardware OpenAL, so would behave the same as hardware.

Sorry for the weird wording. What I meant here was when Directsound3D title is being wrapped into OpenAL, be it software or hardware, via Alchemy or DSOAL.

You can use a different decoder preset that excludes the center channel <..>

Thanks for the suggested workaround. I tried that and indeed it excludes the center channel. Still, this pertains to all the sounds, including those that would be preferable on the center channel. It also doesn't fight the more overarching issue of music being processed (panned to the rear speakers or otherwise modified), which is always undesirable except the cases when it is in-world, i.e. "physical" jukebox playing a track or a radio or some such.

an app doesn't actually have to "switch to" OpenAL Soft

Again, sorry for the wording. What I meant was previously they packaged vanilla OpenAL32 and wrap_oal which allowed you to either use the hardware solution, or to choose any other installed (including openal-soft) via the system router. Now they're packing openal-soft renamed as OpenAL32 (or something along these lines). I think that what they do has the music stream outside the 3d audio scene.

Now, to get the issue back on track, I've carried some comparative tests between the real hardware and openal-soft. Before testing, I made sure that there are no system audio enhancers running, no CMSS3D or any other such stuff. I also checked in game logs that OpenAL subsystem was properly initialized and 3D positional audio is working (e.g. all the speakers are working, reverb and positioning is in place).

All the titles are native OpenAL (in order to exclude DS3D translation quirks):

			ct_oal+SB X-Fi						soft-oal
Prey		Music is in FL and FR only		Music is always upmixed
Unreal 2	Music is in FL and FR only		Music is in FL FR only in menu, but in-game is upmixed
UT2004		Music is in FL and FR only		Music is always upmixed
El Matador	Music is in FL and FR only		Music is always upmixed

I know the sampling is low, but I'll try to extend that list.

Still, it seems, the developers didn't intend for music to be processed, and they had some sort of mechanism to avoid that (and most of them used it). Could it be that current EAX emulation omits such a feature for such kind of sources? I mean, sources that are not meant to be processed in any way.

[Zoomer1988]

I wonder, maybe this is a question to the EAX guru @bibendovsky?

Would there be some sort of characteristic, i.e. some sort of "flag" game developers would use to identify that the specific sound should be played as stereo on appropriate speakers and not processed in any way, when working with OpenAL. Maybe it is possible to detect such markers and play such sounds and music outside the virtualized environment and avoiding the speaker panning using the AL_SOFT_direct_channels[_remix] extension, as @kcat suggested?

I think, this is quite a relevant feature for not only surround sound systems but for the headphone users too.

The former would benefit from much more faithful music reproduction, since it's often the case the front stereo pair does not require the use of a subwoofer at all where the satellite channels almost always rely upon it, the stereo separation won't suffer, and at the same time the majority of users of such a system wouldn't want any music on the center channel anyway since that speaker definitely is not designed for music at all.

The latter will enjoy better stereo separation, for, I assume, any sort of HRTF filtering would negatively impact it as there's no additional metadata to achieve proper 3d virtualization. Not to mention that any HRTF shenanigans affecting frequency response or altering the source in any way wouldn't be appropriate for a bog-standard stereo music recording, if only we're not trying to emulate some sort of presence in a concert hall or something like that.

[kcat]

Would there be some sort of characteristic, i.e. some sort of "flag" game developers would use to identify that the specific sound should be played as stereo on appropriate speakers and not processed in any way, when working with OpenAL.

There isn't. In those days, multi-channel sounds would be played directly on output speakers with no need for a flag because it would primarily be stereo music tracks, or maybe surround sound for a video/movie. DSound wouldn't even allow EAX to be applied to multi-channel/non-3D sounds (a buffer needs to be marked as a hardware buffer to get hardware/EAX processing, a buffer couldn't be set as hardware without the CTRL3D flag, and that flag couldn't be used with non-mono formats). But these days with HRTF, virtualized mixing beds, and multi-channel sound panning, you can't just play multi-channel sounds directly on the output without potentially degrading audio quality. The option is still there with the aforementioned extension, but it needs to be used with some care (you wouldn't necessarily want to use direct channels with a 7.1 sound on 5.1 output, or with a non-binaural sound on headphones/HRTF, or with a sound that's intended to move around at some point).

The latter will enjoy better stereo separation, for, I assume, any sort of HRTF filtering would negatively impact it as there's no additional metadata to achieve proper 3d virtualization.

The problem with playing random stereo sounds on headphones is actually that they get too much stereo separation, creating an unnatural effect where left-panned sounds are put directly in the left ear with nothing on the right (and vice-versa), and center-panned sounds being interpreted as coming from inside the head. That hyper separation with inside-the-head panning causes listening fatigue. The virtualization with HRTF creates an effect that mimics what other people hear when using speakers, providing a more consistent auditory experience that's less draining on the listener.

Even without HRTF, there are tools and techniques intended for making stereo sounds more 'headphone friendly' by crossfeeding sounds between the channels in a somewhat natural way, with a delay and attenuation, which helps alleviate listening fatigue. The only time it works well to play a stereo sound directly on headphones is when the sound is designed for headphones, that it has virtualization or crossfeed filters baked into it (which in turn means it's not as well suited to speakers).