Switching playback streams between devices seamlessly on windows #533
Comments
|
Anyone? |
|
Option 2 would be possible, but with caveats. There would be extra latency since there would be the source streaming buffer queue on top of the device buffer OpenAL writes to, and you can't really match the rendering format to the device format, since OpenAL doesn't expose the latter (aside from maybe detecting HRTF/headphones with the You could use SDL for cross-platform audio output, with an OpenAL Soft loopback device to render/mix. Then to switch devices, close and open the SDL device and continue rendering with the same loopback device (first changing its format with I do intend on adding a feature to be able to "reopen" a device, or move a playback device to another output. There's just currently some dependencies between the device handle and the output handler that would make it dangerous, which would need to be handled first. |
hmm, not sure i understand this part. i would be passing the loopback output to openal anyway, so wouldn't this be identical to the situation where i'm not using loopback and loading audio assets into openal to play that may or may not match the device format? i.e. what's the difference between egs. specifying & loading a float 32 bit audio asset into openal (regular initialization w/ hardware device), and calling play on it, as opposed to rendering loopback output to float 32 bit and then passing that output to an openal streaming source on the hardware device context, while specifying it as float 32 bit? Isn't OpenAL going to be handling the matching of the audio format to the device format under the hood in both cases?
Any chance there's a very rough estimate of this timeline? |
I mean having the loopback device render to stereo, quad, 5.1, 71, or whatever depending on the actual output device. Ideally the loopback device would render to the output device configuration, and the openal playback device's streaming source would enable direct channels (to avoid extra panning virtualization on the channels of the final mix). The only way I can see to avoid that with an openal playback device is to render the loopback device to b-format and stream a b-format source on the playback device. Though that may have its own slight issues with stereo speakers and headphones (the positioning might not be quite as good). OpenAL isn't really well designed for simple direct stream output.
Maybe I could get something that sort of works by early next week, perhaps. But there will almost certainly be issues with backends that directly access hardware, with hardware that can't open an output multiple times simultaneously. It's not a very typical setup, as most people will have a software mixer/server to handle multiple outputs on the same device, if not a device capable of hardware mixing, but they do exist. |
|
Suffering from the same conundrum here. I have implemented just that; having the normal 3D audio processing in a loopback device and then a transient physical device with a single Another thing I have also noticed is that changing the default output device without unplugging or touching the previous default doesn't really disconnect Once I'm notified of that I attempt to I wish there was a cleaner way of doing this. Callback-based, if possible. Reusing Ultimately, I think at least in extension-less/plain OpenAL the best way of doing things would be to have a virtual device that matches the format and layout at startup and retains them as devices and formats change underneath. You may have to restart the application to get 5.1 and if the opposite thing happens you'll get downmixed audio, but at least you don't get pure silence on random changes until restart, like now, which I think is worse in real-world usability terms. Let me know what you think. -- |
@kcat I'm confused why you're suggesting a b-stream format render to a b-stream format source. Can you explain what you mean by 'avoid extra panning virtualization' ? Currently, my hardware device format is stereo (I'm just using a laptop with headphones/speakers). So I've set up the loopback device to render to stereo, and the streaming source on the openal playback device is also stereo. Am I going to be losing some quality of spatialization this way? I thought stereo sources were untouched/not spatialized...
@Swyter can you explain what |
|
Yeah, I had to do a bit of spelunking in the Git history and the various internal tools that use them.
ALsizei pcAudioCallback(ALvoid *userptr, ALvoid *sampledata, ALsizei sizeinbytes)
{
/* swy: render an arbitrary amount of samples from the loopback device,
relay them to the physical/transient one via callback */
alcRenderSamplesSOFT(alDevLoopback, sampledata, sizeinbytes / frameSize); return sizeinbytes;
}
/* swy: create a transient device to relay whatever the loopback device is playing.
save the name at startup to detect changes in the default device via string
compare, not ideal, but the OpenAL Soft doesn't seem to refresh this list
on its own, and no AL_EVENT_TYPE_DISCONNECTED_SOFT thingies are fired */
alDevPhysicalName = (char *) alcGetString(NULL, ALC_DEFAULT_ALL_DEVICES_SPECIFIER);
alCtxPhysical = alcCreateContext(alDevPhysical, attrs);
alcMakeContextCurrent(alCtxPhysical); AL_ASSERT;
ALuint LoopbackBuffer = 0, LoopbackSource = 0;
alGenBuffers(1, &LoopbackBuffer);
alGenSources(1, &LoopbackSource);
alBufferCallbackSOFT(LoopbackBuffer, AL_FORMAT_..., 48000, pcAudioCallback, alDevPhysical, 0);
alSourcei (LoopbackSource, AL_BUFFER, LoopbackBuffer);
alSourcei (LoopbackSource, AL_SOURCE_RELATIVE, AL_TRUE); AL_ASSERT;
alSource3i(LoopbackSource, AL_POSITION, 0, 0, -1); AL_ASSERT;
alSourcei (LoopbackSource, AL_ROLLOFF_FACTOR, 0);
alSourcei (LoopbackSource, AL_LOOPING, AL_FALSE);
if (hasBypassExtension)
alSourcei(LoopbackSource, AL_DIRECT_CHANNELS_SOFT, hasBypassRemixExtension ? AL_REMIX_UNMATCHED_SOFT : AL_DROP_UNMATCHED_SOFT); AL_ASSERT;
alSourcePlay(LoopbackSource);
alcMakeContextCurrent(alCtxLoopback); AL_ASSERT;
In the game I work on I originally tried to mirror this extension using plain OpenAL buffer-chaining and it was imperfect and extremely challenging, so I can see why Implementing a less abstracted ring buffer, where every bit can be flipped, with a more abstracted way of doing the same thing via a collection of in-flight, fixed-size buckets is like trying to do surgery with a rubber ducky. Giving the implementation a correct approximation of what the current offset in the virtual "buffer" chain is wasn't very fun either. But I'm a pretty bad programmer. |
When you render using, say, a 5.1 loopback device, then play a 5.1 source on the playback device, OpenAL will virtualize the source channels. Each channel will be given a relative position around the listener, where it'll be run through the usual panning algorithm to be positioned as best it can regardless of the playback device's actual output configuration. So if the user has a quad or 7.1 setup, each source channel will seem to come from the same general position (rather than 5.1's side channel being naively moved to quad's rear channel, which isn't in the same position). Or if the user has stereo output, the channels will be downmixed appropriately. Or if the user has HRTF enabled, it will create a virtual surround sound effect. So not only will sources be panned virtually when rendering the mix with the loopback device, the loopback device's channels will be panned virtually when mixing with the playback device. Using a B-Format loopback mix (and a B-Format source stream) can help. B-Format doesn't render discrete channels that need to be positioned, it instead renders a continuous 3D soundfield. A B-Format source decodes that soundfield to the given output speaker configuration. Internally, this is exactly what OpenAL Soft's playback devices do*. It mixes to B-Format, then decodes the B-Format mix to the user's speaker configuration. A B-Format loopback mix then avoids that last decode bit, and the B-Format source stream completes it, avoiding the extra virtualization while matching the user's output configuration.
A stereo loopback mix with a stereo source playback should work out fine. But once you get into other things, like surround sound or HRTF, you'll have work ahead of you to get separate loopback and playback devices working optimally.
More or less. The callback gets invoked in the mixer when the source needs some more samples to mix. It requests exactly as many samples as it needs when it needs them. There's one big caveat: since it runs in the mixer, the callback needs to be fast and real-time safe. It can't do any locking, I/O, blocking, waiting, or make any calls that could do these things (or otherwise take too much time and cause the audio update to miss its deadline). And technically speaking, I've been thinking of relaxing that about |
|
Thanks a lot for the explanation, that cleared up a few things I was wondering about. I'll give b-format a go. What do you think about the idea of a virtual device that wraps all this complexity in a less roundabout way? I think it could simplify hot-plugging for most non-professional-audio users that can't tear their long-running 3D context down. I call I'm considering using something lightweight and unbuffered like |
|
Side note: funnily enough even Minecraft seems to have this problem: https://bugs.mojang.com/browse/MC-44055 |
If it's done at the system level, where the app isn't responsible for moving its output around to different devices, or have to worry about reconfiguring its output, that's more or less fine. As long as you're okay with the fact that the OpenAL device configuration won't update with the move (e.g. if you move from a stereo device to surround sound, OpenAL Soft will continue mixing stereo, and if it moves to headphones, it won't start using HRTF automatically). The main issue with doing virtual devices in OpenAL Soft itself basically boils down to the device needing to be reconfigured for the new output, as there's no guarantee a device it opens will be able to match what it's currently using. And the device can't/shouldn't asynchronously reconfigure itself (an app should be reasonably assured things like the HRTF status or the refresh/update rate won't randomly change while in use). This is partly why Probably what I'll do is add some kind of messaging log/queue that can be queried with |
|
@Swyter how do you get access to |
It's an in-progress extension right now, subject to incompatible changes, so it's not made public. The current definitions need to be copied out of Once the extension is finalized, its definitions will be moved to the public |
|
There's a function/in-progress extension now with commit e824c80 for reopening a playback device. if(alcIsExtensionPresent(device, "ALC_SOFTX_reopen_device"))
{
ALCboolean (ALC_APIENTRY*alcReopenDeviceSOFT)(ALCdevice *device, const ALCchar *name, const ALCint *attribs);
alcReopenDeviceSOFT = alcGetProcAddress(device, "alcReopenDeviceSOFT");
success = alcReopenDeviceSOFT(device, new_device_name, attribs);
}
If the call returns Be aware that this is an in-progress extension (as noted by the There isn't yet a way to determine if the default device changed or anything, although as an alternative, you can use WASAPI directly just to get notifications for devices being added/removed and the default device being changed, and use OpenAL's reopen device function as needed in response to it. Obviously that will only work on Windows, but it's an option until I get something more cross-platform. |
|
This is fantastic, thank you very much! Makes sense. I'll give it a go. I understand the complexity of having to provide a cross platform way of getting change notifications efficiently, when half of the backends are pretty simple and may not map well, or the solution needs a bunch of workarounds. -- Having an event queue and expanding the Just eventually having That and virtual devices aside, I appreciate the work you are single-handedly doing by keeping OpenAL relevant and modernizing a few blind spots that (understandably) haven't aged well. After using it out of the box in four different platforms (and doing my own backend on Nintendo Switch) it has always been dependable and rock-solid. So no complaints, I know what I'm going to get. |
|
Small note: For those reading this thread, and for future reference; keep in mind that profiling these So, careful with your |
Interesting, I'm surprised it's that fast. It has to marshal the call to a separate message handler thread (because Windows can't make it easy to call WASAPI methods on a whatever thread a caller happens to be on because of COM restrictions), and then has to wait for the message handler thread to wake up and do the actual enumeration before signaling that the enumeration is complete, before the calling thread can wake back up and report back with what it found. I'd have expected the scheduler to take a bit more time to wake up the handler thread to handle the enumeration, and to wake the calling thread back up after enumeration finished. |
|
Thanks for the quick turnaround on this kcat! I've been trying this extension, and it only seems to work sporadically. Often times, the audio stream is not moved (even though Is there any more info I can provide to help with debugging? |
|
From what I can see,
When you say "the audio stream is not moved", do you mean it keeps playing on the same device, or outputs stops completely? Does it play at all on the new device? Since If you wait a second or so and call |
|
Ok first let me clarify my process, which is as follows: I have my headphones plugged into my laptop. I fire up my audio engine (which uses OpenAL), and verify I can hear audio playing through my headphones as usual. Then, I unplug my headphones. I expect the audio output to now continue through the laptop speakers. This happens around 25% of the time; most of the time I get that padding error in the console, and nothing plays through the speakers (even though I'm getting the notification of the headphones being unplugged from the Windows WASAPI layer btw, and using that to trigger the
I can't answer the part of 'does it keep playing on the same device`, because the headphones are unplugged, so I'm not sure how to verify that. And as mentioned above, output stops completely i.e. i don't hear anything on the speakers.
This seems to make things a bit better i.e. the speakers play the audio a bit more reliably, maybe 75% of the time now, but it's still not consistent (I wait 2 seconds before calling EDIT: Interestingly enough, if I don't wait for 2 seconds and just call |
Are you also able to get notifications about devices being reconfigured? It sounds like after the headphones are unplugged, the
That is really odd. Just to be sure, when you get the padding error, the ALCdevice becomes disconnected, right? And the audio plays on the new device for as long as it doesn't become disconnected? And after it becomes disconnected, Either way, I made a slight tweak with commit 743f093 that might, unlikely but possibly, work better at reopening a device. It would be helpful if you can test. |
|
sorry for the extended pause here; i got pulled into another project. so i tested out the latest version of openal-soft and unfortunately the problem still doesn't seem to have gone away completely i.e. I will occasionally get the padding error. It seems like it's happening less often now, but it's hard to say that definitively.
Not sure what you mean by reconfigured per se, but I have a device monitor service that will fire a notification when a device is added, removed, or the default device has changed.
How can I check to see if the ALCdevice has become disconnected?
The audio does not play on the new device when I get the padding error. So if I had headphones plugged in (audio playing through them fine) and I unplug them and get the padding error, the audio does not play on the speakers ("new" device) at all.
Yes, this is correct. It seems I spoke too soon earlier when I said that calling Please let me know how else I can continue to help testing, thanks |
I mean if the device changes its format or period size, or switches between enabled and disabled (if that's different from added/removed), and things like that.
You can either use the
Okay, so it never gets a chance to start every time the error happens. The error is never a delayed occurrence. How about if, after calling |
Just tried this, and waiting for 5/10/20/30 seconds does NOT make a difference. When I hit the padding error, it doesn't matter how long I leave the device alone before calling
I can now confirm that yes, the device is detected as having been disconnected. I did To re-iterate the sequence of events in the error case:
Note that, however, when I don't get the padding error (i.e. and when audio is successfully transferred to the speakers), |
That final Getting a padding error just as the headphones unplug is expected. The audio device becomes disabled, which reasonably causes an At this point, calling Are you able to provide a full trace log of a run where audio failed to come back on after unplugging the device? Either wherever the output of |
|
Here's the log using
Let me know if you need any more info. |
|
If that's where the log ends until the device closes, everything looks fine. The new device is opened, configured, and started without error. Nor is there any error from the new mixing thread. Is the device still connected (as reported by |
No, the device is not reported as connected when it's silent (
No, just checked, and the source states become I also got a trace log when I don't get the padding error, and it's identical to the one I posted above for the padding error, with the exception of this line, which is present in the padding error log above, but absent in the successful case log:
This line appears twice in the padding error log (once after Does that mean anything? |
Just to make sure, this is the case after calling
That would suggest the device never became disconnected. Which would be odd if you don't reopen the device until the device disconnects. Are you able to make a test case that exhibits the problem? A minimal example with code that waits for the device to disconnect and reopens the device, showing that it's still disconnected afterward? |
No, this is the case before calling I just tried querying Also to address what you said here earlier:
This is not what I'm experiencing though. I only get the padding error when hitting the silent audio case. Most of the times when I unplug my headphones, there are absolutely no errors (neither the padding nor the buffer data one). Do you still want me to make an example with code? In what form would you like this? Should I just paste a minimal set of code here that reproduces the problem for me? |
Sure, but the problem you're having is that there's still no audio after reopening the device. There obviously won't and can't be any audio when the headphones are unplugged, which causes
It's expected for the device to become disconnected when the headphones are unplugged (which is why the padding error shows up). The I think you might be getting confused. When you physically unplug the headphones, Windows detects this and marks the system device as disabled, changes the default audio device as needed, and sends out a notification about the change. After that, the next time OpenAL Soft tries to access that system device it gets an Because the sources were previously stopped when the headphones were disconnected, they won't be playing when it reopens/reconnects and it will stay silent if you don't play a source. If, when you do play a source now, it becomes |
|
Before I post a detailed reply, let me clarify:
Are you saying that, after calling |
All stopped sources that should be playing, yes. You of course don't need to play sources that could've already stopped naturally, or sources that you don't want to start over from the beginning. |
|
Okay, I think I see where our misunderstanding is then. The key lies in what you said here:
Specifically this part (which I'm assuming is asynchronous?):
Ok, as you know, when I unplug the headphones, my windows device monitor service sends a notification about the device change, and that's when I invoke my callback that calls Scenario INow: most of the times, when my callback gets invoked, it appears that OpenAL Soft has not yet tried to access that system device. This is evidenced by the fact that a) I don't get any padding error, and also b) at this point (before calling Scenario IINow, if "the next time OpenAL Soft tries to access that system device" happens before my callback is invoked, that's when I get the padding error. In this situation, if I query --
So nope, this isn't (and wasn't ever) happening (I was simply observing the sources were It sounds like, however, that in order to 'seamlessly' continue playback on the speakers -- i.e. have my sources simply pick up where they left off, and continue playback on the speakers from the offset they were at when they were playing back on the headphones -- I would need to continuously be caching what offset my sources are at in every instant. That way, when OpenAL Soft stops all of them, and I have to call At this point I'm inclined to ask -- is it possible to have Let me know if you have suggestions for how I can resume playback on the speakers at the last playback point in a way that has minimal overhead. |
Not actually pause, no (there's too much of an assumption that sources can only ever stop asynchronously, and can only pause synchronously). But it might be possible to have them hold their play state, effectively pausing but still report as playing despite not actually doing so, and automatically start playing again when reconnected. It would have to be opt-in behavior, probably a context creation attribute or a context state, so an app would only set it when it knows it can't rely on a playing source to always be progressing. Although it's notable that such a thing is only possible because OpenAL Soft does all the mixing itself. If processing were to ever be offloaded to an external process or device, and that process or device gets lost, then the current play state would be well and truly lost. I don't foresee OpenAL Soft ever doing that, but it's something to keep in mind regarding other potential implementations.
No, all those properties remain as they were. It's just the play state (number of processed buffers and byte/sample/sec offset included) that's lost. |
That would be very helpful. Any chance this is functionality that you plan on committing anytime soon?
So is there any other recourse I have currently other than saving the offset positions continually and restoring them? |
Likely some time very soon. Within they day, probably.
Not if you change the source's pitch or the source or listener velocity (which implicitly changes its pitch). If you don't, then if accuracy isn't much of a concern there is the option to just save the source's start time when you play it, then on disconnect, get the last known device time and use the difference. So when starting a source, something like: alSourcePlay(source[i]);
// From ALC_SOFT_device_clock
alcGetInteger64v(device, ALC_DEVICE_CLOCK_SOFT, 1, &source_start[i]);This gets an alcGetIntegerv(device, ALC_CONNECTED, 1, &connected);
if(!connected)
{
int64_t stop_time;
alcGetInteger64v(device, ALC_DEVICE_CLOCK_SOFT, 1, &stop_time);
... reopen the device ...
std::vector<ALuint> pending_sources;
for(each source that should be playing)
{
int64_t play_time = stop_time - source_start[i];
int sample_offset = (int)(playtime * source_srate / 1000000000);
// Make sure to wrap sample_offset around the loop points if the source is looping.
alSourcei(source[i], AL_SAMPLE_OFFSET, sample_offset);
pending_sources.push_back(source[i]);
}
// Play all sources that should be playing at once
alSourcePlayv(pending_sources.size(), pending_sources.data());
}This will of course all be unnecessary if you don't mind waiting for the extension to hold the source state. |
|
Thanks for the writeup. Sure, I don't mind waiting for the extension update; please let me know when it becomes available. |
|
Commit 26c8c50 partially implements a new extension for this. Call |
|
Thanks! Going to try this out and will report back.... |
|
@kcat I'm using OpenAL v1.22.2 and extension ALC_SOFT_reopen_device nor ALC_SOFTX_reopen_device is available. Function alcIsExtensionPresent() returns false. Where I can find this extension? |
|
If you're using the router, you need to have an open device handle for an OpenAL Soft device to query and get the function with. A null device pointer will query the router itself, which doesn't handle the extension directly. Also, for it to work with the router, you need to use OpenAL Soft's router. Creative's router is designed in a way that doesn't allow the extension to work. If you're not using the router, then double check you're using that version of OpenAL Soft (with a context set up, |
|
@kcat You are right. I got version mixed up with the system libraries v1.19. I have replaced system libraries with compiled one but now the device will not get opened. That is strange.. |
|
@kcat I have cleared out the problem with the version and the extension is found properly. I didn't tested it though as the device does not change when output sink is switched. I was switching between the speakers and the bluetooth headset. In v1.19 the device was connected with audio output and it changed when switching between the speakers and bluetooth headset. That is why I was searching a way to reuse the buffers in the new device and found out this thread. In v1.22 I see that OSS backend is used and changing the audio output sink does not change the device name anymore. So the problem is solved without the need to use the extension. |
Hi,
I'm wondering if it's possible to switch playback streams seamlessly between devices on Windows. For egs, say I have my headphones set as the default device, and I have an active OpenAL session running and outputting audio to this device. While this is happening, if I set my speakers as my default device, I would like the OpenAL audio output to switch to the speakers seamlessly. My current understanding is that, this isn't possible out of the box, and my 2 options are basically:
To manually implement, in my app, a way to preserve all the OpenAL buffers, sources, playback positions, context etc. , and manually close the headphones device, open the speakers device, and re-create the entire OpenAL session at the preserved position.
Alternatively, I could use some sort of virtual device to actually output my audio to, and in turn route the output of that virtual device to the windows audio device of my choosing. My question here is -- can I use the OpenAL loopback device for that purpose? i.e. always, by default, render all my audio output to the loopback device. Simultaneously, I would open the Windows default device (headphones) and, render the loopback output and pass that rendered output to an OpenAL buffer/source that would be playing out on the headphones device. This way, when the windows default device changes to the speakers mid-session, I would close the headphones device on OpenAL and open the speakers device instead (all this time while keeping the loopback device active and running), and simply continue to pass the loopback rendered audio output to the new (speakers) device instead. This would save me the trouble of implementing a system in my app that would have to manually preserve the buffers/sources/context of the OpenAL session at all times.
Does this make sense? Would # 2 be possible or am I missing something?
Thanks
The text was updated successfully, but these errors were encountered: