Proposal to expose backend-specific functionality

[roderickvd]

Proposal to expose backend-specific functionality in cpal

This is a proposal for how cpal could expose host- and platform-specific functionality. I'd like feedback before we commit to it, hoping to get it right the first time.

Goal

The goal is to have cpal support for functionality that only some backends offer, without pushing users into #[cfg]. Examples:

• open the ASIO control panel (ASIO: open control panel #1074)
• WASAPI raw capture, to bypass the OS effects chain (WASAPI: Raw stream mode #1173)
• set PipeWire stream metadata so a player shows up properly in the graph (Pipewire: API to set identifying properties #1186)
• name the JACK client instead of taking our default (can't set the name of jack client #1235)
• exclusive mode on WASAPI and CoreAudio (Support Exclusive Mode #106)

The constraint is that a user holds a type-erased cpal::Device / cpal::Stream / cpal::Host, because cpal exposes a cross-platform API without having users to put #[cfg] everywhere. Whatever we add has to follow that. Code that reaches for an ASIO feature should still compile on Linux and just do nothing there. This is the "cp" in cpal.

In StreamConfigBuilder (#1010) I took a stab at this with one cross-platform builder carrying every backend's knobs through on_alsa(...)-style closures. I think it got the cross-platform no-op right, but it treated every knob as the same kind of thing when they aren't. Some are stream-, device-, or host-level, and they differ in how many backends share them. The below is about sorting that out.

Proposal

What it could look like:

use cpal::platform::ext::*;   // new: extension traits prelude
use cpal::traits::{DeviceTrait, HostTrait, StreamTrait};

let device = cpal::default_host().default_output_device().unwrap();   // existing
let config = device.default_output_config()?.config();                // existing

let stream = device
    .output_stream(config)                      // new: builder; old build_output_stream remains
    .exclusive(true)                            // new: errors if the backend can't do it
    .auto_connect(false)                        // new: ignored where it doesn't apply
    .on_alsa(|a| a.mmap(true))                  // new: ALSA-only, skipped elsewhere
    .build(data_cb, err_cb, None)?;             // new: sample type from the callback
stream.play()?;                                 // existing

if let Some(asio) = device.as_asio() {          // new: None on other backends and platforms
    asio.open_control_panel()?;
}

Second, an escape hatch to get the native type by consuming the CPAL device:

use cpal::host::wasapi;   // new: concrete host type, becomes public

// consumes the cpal::Device; a failed conversion hands it back as the error
let wasapi_device: wasapi::Device = device.try_into()?;
let imm: IMMDevice = wasapi_device.into();

Rationale

After #1010 I realized that one mechanism cannot cover all of this. It matters how widely supported a certain capability is:

• Core: most or all backends. What DeviceTrait / StreamTrait already are: enumerate, query configs, build, play/pause, buffer size.
• Multi-backend: a few that share the same capability. Exclusive access, auto-connect, raw capture, buffering depth, naming, and metadata.
• Single-backend: unique backend functionality. ALSA access mode, ASIO control panel, Android's input-preset. A backend-specific functionality not present on others.
• Native: the raw handle. IMMDevice, snd_pcm_t, jack_client_t. An escape hatch you reach only by taking the cpal handle.

The other axis is when and where: you can't open the ASIO control panel from a stream builder, for example. JACK's client name is the reverse: a plain value, fixed before there's a Device to call it on.

Examples

Not a complete list:

Functionality	Backends	Tier
exclusive access	WASAPI, CoreAudio (hog), AAudio	Multi-backend (guarantee)
raw capture	WASAPI, AAudio, PipeWire, (iOS)	Multi-backend (guarantee)
auto-connect	JACK, PipeWire	Multi-backend (hint)
buffering depth (period/buffer count)	ALSA, JACK, AAudio, PipeWire	Multi-backend (hint)
naming and metadata (media.name, etc.)	PipeWire, PulseAudio, WASAPI	Multi-backend (hint)
access mode (MMap/RW), (de)interleaved	ALSA	Single-backend (build-time)
input preset (source, AEC)	AAudio	Single-backend (build-time)
control panel	ASIO	Single-backend (action)
client name	JACK	Multi-backend (construction, hint)
native device/stream handle	each	Native

About a few of these:

Exclusive access

ALSA can be shared (dmix/dsnoop/default) or exclusive (hw:), but you pick that by which PCM you open, not by a flag at stream time. The other three decide it as an open-time flag on the same device. So the shared exclusive verb means "decide exclusivity when the stream opens", which fits WASAPI/CoreAudio/AAudio but not ALSA or JACK.

Naming

PipeWire and PulseAudio carry it as stream metadata, WASAPI as the session display name in the volume mixer. JACK is the odd one out only in when it binds. Its client name names the whole connection and is fixed when the client is created, so on JACK it's a hint on the host builder instead of the stream builder (see below); elsewhere it's per-stream.

Buffering depth

Size is already Core (BufferSize). Depth is how many buffers deep you go past the default double-buffering. ALSA periods and AAudio bursts take it per stream. JACK's nperiods is fixed when the server starts. PipeWire's quantum is graph-wide, so there it's a no-op. WASAPI and CoreAudio give a size or duration, not a count. Since it's really an extension of buffer size, we should fold the control into the BufferSize range refactor (#447).

APIs

Core: Stream builder

Build-time knobs need somewhere to hang off, so a output_stream() builder gives them one. It'd be purely additive: build_*_stream is unchanged.

// new: on DeviceTrait
fn output_stream(&self, config: StreamConfig) -> OutputStreamBuilder<'_>;

impl OutputStreamBuilder<'_> {
    // typed terminal: sample type from the callback, like build_output_stream
    fn build<T, D, E>(self, data: D, error: E, timeout: Option<Duration>)
        -> Result<cpal::Stream, Error>;

    // raw terminal: runtime format, like build_output_stream_raw
    fn build_raw<D, E>(self, sample_format: SampleFormat, data: D, error: E,
        timeout: Option<Duration>) -> Result<cpal::Stream, Error>;
}

build is typed and reads the sample type from the callback, build_raw takes a runtime format, mirroring today's build_output_stream and build_output_stream_raw. Input gets its own InputStreamBuilder, so an inapplicable knob doesn't exist on the wrong builder rather than failing at runtime.

Core: Host builder

Host-level knobs need the same thing stream knobs got: an erased builder to hang off. Host::builder() mirrors Device::output_stream(); cpal::default_host() is unchanged and stays the default for the no-knobs case.

// new: erased, portable
fn cpal::Host::builder() -> HostBuilder;

impl HostBuilder {
    fn build(self) -> Result<cpal::Host, Error>;
}

Multi-backend: Knobs

Extension traits named after the knob they represent, implemented for the erased builder. The knob is always there, so it compiles everywhere; whether it's honored is settled at build().

pub trait ExclusiveAccess: Sized {
    /// Request exclusive access. A *guarantee*: `build()` errors with
    /// `UnsupportedOperation` where it can't be met, so you never silently get shared.
    fn exclusive(self, exclusive: bool) -> Self;
}

pub trait AutoConnect: Sized {
    /// Auto-connect this stream's ports to the system graph. A *hint*: ignored where
    /// there's no graph. Defaults to `true`, matching today's behaviour.
    fn auto_connect(self, yes: bool) -> Self;
}

A guarantee like exclusive has to error when it can't be met, because silently handing back a shared stream is explicitly not what the caller ordered. A hint like auto_connect should just no-op where it makes no sense. Each knob has to say which it is in its docs.

Guarantees should have a read-only probe on Device, like supports_exclusive(), so an app can branch before building. Hints don't need one: they would no-op, so calling them is always safe.

The same shape works one level up. ClientName is a hint on HostBuilder instead of the stream builder, since JACK's client is created there:

pub trait ClientName: Sized {
    /// Set the client/application name shown at connection time. A *hint*: only JACK
    /// honors it today; ignored elsewhere. Defaults to the crate name.
    fn client_name(self, name: &str) -> Self;
}

If you already have the concrete jack::Host before erasure, its own client_name setter does the same job directly. Nothing needs both.

Single-backend hooks

This is #1010's closure, kept for where it actually fits: one backend's unique build knobs. An extension trait with an options type that's a PhantomData stub when the backend isn't compiled, so it still names and compiles everywhere:

pub trait AlsaStreamBuilderExt: Sized {
    fn on_alsa(self, f: impl FnOnce(AlsaStreamOptions) -> AlsaStreamOptions) -> Self;
}

impl AlsaStreamOptions {
    pub fn mmap(self, yes: bool) -> Self;
}

.on_alsa(|a| a.mmap(true)) runs the closure only when the builder is actually ALSA; otherwise it's skipped.

Single-backend accessors

Type-gated accessors for things that aren't stream config at all, like opening a control panel:

pub trait AsioDeviceExt { fn as_asio(&self) -> Option<AsioDevice<'_>>; }

impl AsioDevice<'_> {
    pub fn open_control_panel(&self) -> Result<(), Error>;
}

as_asio() returns None when the device isn't ASIO, or when ASIO isn't compiled in. When it's Some(asio), call asio.open_control_panel()?.

Native handles

If you genuinely need the native handle, you can get one using a consuming conversion. By leaving nothing for cpal to manage, we prevent state mismatches and other foot-shooting:

// consumes the cpal::Device; a failed conversion hands it back as the error
let wasapi_device: cpal::host::wasapi::Device = device.try_into()?;
let imm: IMMDevice = wasapi_device.into();

So concrete host types can be public, and the only way to get one is to consume the managed handle.

Alternatives considered

One builder for everything, like #1010

Where I started. The closures are nice and the cross-platform no-op is right, but it treats a control panel, an exclusive-mode request, and an ALSA access mode as the same kind of thing, when they are not.

Flatten them onto a single builder and the shared ones end up spelled per-backend (on_wasapi(exclusive), on_coreaudio(exclusive), and so on) when they should be one verb, and the actions don't fit a config closure at all. I kept the on_alsa hook, but moved it to Single-backend where it belongs.

Get the backend type, then build through it

This came up in later sketches on #1074: device.as_wasapi()?.build_*. It's fine for an action, but routing stream construction through a fallible accessor means ordinary cross-platform building now goes through as_wasapi()? and a match for no real gain. Building is Core and should stay on the device. The accessor is there only for the genuinely single-backend actions.

Expose DeviceInner and derive TryUnwrap

Like try_unwrap_wasapi_ref(). Cheapest but makes the enum we use for type erasure part of the public API, and reintroduces #[cfg] where the feature is not compiled in. It saves us very little while adding cost to users.

#[cfg]-gate the accessors

Honest about what's available, and you could argue app code already branches on the host anyway. But it puts #[cfg] back at every call site, which is the one thing the platform enum exists to spare users.

[Marco-Farruggio]

I'll post mine here aswell, and then get started reading yours :)
A few notes overall, a lot of my example code excludes error handling,
and most of the names ive chosen are simply for clarity reading the code now,
and are by no means suggestions of the real names.
Also, this is an API sketch, so some things may still need ironing out.

API Options

Option 1

Notes:
Pros:
Cons:

Definition

#[cfg(all(target_os = "windows", feature = "asio"))]
pub trait AsioDeviceExt {
    fn asio_open_control_panel(&self) -> Result<(), BackendSpecificError>;
}

// The IMPLEMENTATION for Device is ALSO gated with the exact same condition
#[cfg(all(target_os = "windows", feature = "asio"))]
impl AsioDeviceExt for Device {
    fn asio_open_control_panel(&self) -> Result<(), BackendSpecificError> {
        // Because of this gate, we can safely assume `DeviceInner::Asio` exists here.
        if let DeviceInner::Asio(ref d) = self.as_inner() {
            d.open_control_panel()
        } else {
            Err(BackendSpecificError { description: "Not an ASIO device".to_string() })
        }
    }
}

Usage

let host = cpal::default_host();

let device = host.default_output_device()
    .expect("No output device available");

match device.asio_open_control_panel() {
    Ok(_) => println!("ASIO Control Panel opened successfully!"),
    Err(e) => {
        // the device isnt an ASIO device,
        // OR
        // the method call itself returned some error,
        // i think there should be a clear differntiation between these,
        // but a nested Result<Result< seems like a very bad idea,
        // i think Option<Result< makes more sense, or if we were to flatten
        // to simply one Result<>, maybe a new ErrorKind::WrongBackend 
        // (or some better name for that enum varient)
        //
        // (afterthought), if it were nested (with either Option or Result), i spose that meant you could do
        // some 'safe' unwraps(), such as if youd just polled the asio host's
        // list of devices, you know you could call match .open_control_panel().unwrap() { Ok(()) => {}, Err(e) => {do_smthn}}
        // for the actual error trying to open the panel.
        eprintln!("Failed to open ASIO control panel: {e}");
    }
}

Ok(())

Option 2

Notes:

• these wouldnt even actually have to be written as extension traits,
  they could just be config gated methods, (see Option 3)
  Pros:
• every single method on the backend-specific device wouldnt have to return an "error, this device isn't from X backend"
• only one failure point to expose the inner device
• means each host can simply define all their methods on their Device and expose any ones which should be public facing as pub()
• custom hosts don't have to edit cpal source code to impl their own host

Cons:

• preferably, each host would need to name their own device, such as "AsioDevice", instead of simply asio::Device, maybe
  a macro could handle this? though on second thought im not sure how/if that would work with custom hosts

Definition

#[cfg(all(target_os = "windows", feature = "asio"))]
pub trait AsioDeviceExt {
    // return an Option<> instead of a Result<>,
    // as the only error i could see occuring at
    // this phase would be that it isnt from that
    // backend, and it's instead from another one
    fn as_asio(&self) -> Option<&AsioDevice>
}

#[cfg(all(target_os = "windows", feature = "asio"))]
impl AsioDeviceExt for Device {
    fn as_asio(&self) -> Option<AsioDevice> {
        if let DeviceInner::Asio(asio_device) = self.as_inner() {
            asio_device
        } else {
            None
        }
    }

    // also a .into_asio() method
}

Usage

if let Some(asio_device) = device.as_asio() {
    let _ = asio_device.channel_names();
}

if let Some(wasapi_device) = device.as_wasapi() {
    wasapi_device.set_some_flag(true);
}

// this would be a compilation error if 
// not gated behind some flag like if
// target_os = linux etc.
if let Some(alsa) = device.as_alsa() {
    alsa.set_another_flag(false);
}

Option 3

Notes:
Pros:
Cons:

Definition

impl Device {
    #[cfg(all(target_os = "windows", feature = "asio"))]
    pub fn as_asio(self) -> Option<AsioDevice> {
        if let DeviceInner::Asio(asio_device) = self.into_inner() {
            asio_device
        } else {
            None
        }
    }

    #[cfg(target_os = "windows")]
    pub fn as_wasapi(self) -> Option<WasapiDevice> {
        if let DeviceInner::Wasapi(wasapi_device) = self.into_inner() {
            wasapi_device
        } else {
            None
        }
    }
}

Usage (Same as Option 2)

if let Some(asio_device) = device.as_asio() {
    let _ = asio_device.channel_names();
}

if let Some(wasapi_device) = device.as_wasapi() {
    wasapi_device.set_some_flag(true);
}

// this would be a compilation error if 
// not gated behind some flag like if
// target_os = linux etc.
if let Some(alsa) = device.as_alsa() {
    alsa.set_another_flag(false);
}

Other Ideas

• A friendly error message:

#[diagnostic::on_unimplemented(
    message = "ALSA is not available on this platform/build",
    label = "no ALSA backend compiled in",
    note = "ALSA support requires target_os = linux (or freebsd/netbsd)",
    note = "if youre on Linux, check the `alsa` feature is enabled in Cargo.toml"
)]

• But more importantly, i started wondering how the stream-specific things would work,
  this is clear:
  AsioHost::some_asio_host_specific_global_method()
  asio_device.open_control_panel()

but how should say opening a stream with exclusive mode work?

wasapi_device.open_stream_exclusive(streamconfig, cb, ecb)
^ this could work, but its very clunky, especially if say 2 variables were involved,

wasapi_device.open_stream_raw(...)
wasapi_device.open_stream_raw_exclsuive(...)

see my point? very bad design idea in my opinion

then there could be
wasapi_device.open_wasapi_specific_stream(config: StreamConfig, exclusive: bool, raw: bool, ...)

but then i thought what if there was like a WasapiStreamConfig, which had .into() / from, for StreamConfig (cross platform)
so you could do wasapi_device.open_wasapi_stream(config: WasapiStreamConfig, callback ...)
which means you could do StreamConfig.into() (auto populates exclusive and raw to false)
(for any host, this .into() would be defined such that it would have no difference compared to the default, asif the
cross platform stream config had been used)

this also opens up a few extra possibilties, as a wasapi's supported configs change from when its running in shared mode,
and your talking directly to hardware with exclusive mode, now we could do:

wasapi_device.supported_wasapi_configs() -> some list of WasapiConfigs,
this list would be the WasapiConfig equivalent of the Configs from a generic_device.supported_output_configs(),
and also WasapiConfig(..., exclusive: true) ones, for when running in exclusive mode.

this does bring issues with how would SupportedConfigRange, and all those structs and things work together, which is quite
complicated, maybe a searcher as i suggested could help here, im not quite sure.

heres an exmaple i had in mind:

let original_config = device.default_output_config().unwrap();

if let Some(wasapi_device) = device.as_wasapi() {
    let new_config = original_config.into().set_raw(true);

    wasapi_device.open_wasapi_stream(new_config, callback, error_callback);
}

also having a .as_host method is nice because then there's only one fail point, so longer things like this can be done:

let original_config = device.default_output_config().unwrap();

if let Some(wasapi_device) = device.as_wasapi() {
    let new_config = original_config.into().set_raw(true);

    let _ = wasapi_device.get_channel_mask();
    let _ = wasapi_device.get_hardware_capabilities();
    let _ = wasapi_device.get_immdevice(); // (just an example here)

    wasapi_device.open_wasapi_stream(new_config, callback, error_callback);
}

[Marco-Farruggio]

I've just finished reading through your notes, very nice, from what I can tell i looked more at how should the api look to a user and you seemed to look at how that would actually work behind the scenes, just a few things, I'm assuming cpal::default_host() will still exist unchanged? take ducking for example, as far as im aware thats wasapi only, and can be implemented on a specific stream, so would that be .on_wasapi(|w| w.ducking(false))? Overall seems much more in depth then mine though, I'd love to contribute some of the code for all this :)

one question going forward though, take WASAPI (again) for an example, how will we handle things like querying exclusive mode supported configs?
maybe something like this?:

if let Some(wasapi) = device.as_wasapi() {
    asio.supported_exclusive_configs()?;
    // this returns the same SupportedConfigRange as normal,
    // but the user knows that they're for exclusive mode, as they just queryed it,
    // which means no new types for it
}