Some questions about host implementations and API design

[LastExceed]

Hello, I'm trying to implement a custom Host, to field-test my new library azo. there are a few things about cpal's API that confuse me:

1. When building streams, the config param includes a channel count, but no information on which channels should be used. does this mean i should always use the first n channels? e.g. StreamConfig.channels = 4 effectively meaning 0..4 ?
2. It seems cpal has no notion of stream data endianness, is this simply left to the user to take care of in the data callback? should I restrict stream formats to native endianness?
3. Should a cpal Host support multiple independent output streams on the same device? if yes, how does channel selection work in this case?

I'm still somewhat new to the audio world, so idk how common (or even reasonable) any of these edge cases are, but I want to make sure that I do my custom host correctly

[roderickvd]

Hello, I'm trying to implement a custom Host, to field-test my new library azo.

Cool, what are your project goals?

1. When building streams, the config param includes a channel count, but no information on which channels should be used. does this mean i should always use the first n channels? e.g. StreamConfig.channels = 4 effectively meaning 0..4 ?

Yeah, channels = N means the first N channels in whatever order the driver reports. There's no channel-map field on StreamConfig (not yet, anyway).

2. It seems cpal has no notion of stream data endianness, is this simply left to the user to take care of in the data callback? should I restrict stream formats to native endianness?

Samples are typed (f32/i16/etc), not raw bytes, so there's no wire format to swap.

3. Should a cpal Host support multiple independent output streams on the same device? if yes, how does channel selection work in this case?

This is backend-dependent and cpal doesn't enforce anything either way. WASAPI and CoreAudio streams to one device get mixed by the OS. ALSA depends entirely on the PCM: raw hw:X,Y will throw a DeviceBusy error on a second open, dmix and others can allow concurrent access. Again, cpal does nothing to enforce channel selection in such cases.

I'm still somewhat new to the audio world, so idk how common (or even reasonable) any of these edge cases are, but I want to make sure that I do my custom host correctly

I can imagine that manual channel mapping may be worthwhile on pro audio devices. JACK and PipeWire can help here.

[LastExceed]

what are your project goals?

The dream is to eventually replace asio-sys in cpal's bundled ASIO implementation, since Steinberg's ASIO SDK can only use a single driver (~device) at a time, and has an annoying license on top of that (azo fixes both of these problems). Granted, they recently added GPL as an alternative license, but I'm not really a fan of copyleft either.

I'm well aware that this is very ambitious, and requires azo to be thoroughly tested "in the field" first, but luckily the custom host API provides a very convenient way to do exactly that.

Samples are typed (f32/i16/etc), not raw bytes, so there's no wire format to swap.

Isn't that precisely what introduces byte order significance? E.g. there is i32::to_le_bytes() and i32::to_be_bytes(). But it seems like cpal's current ASIO implementation doesn't distinguish between them at all:

cpal/src/host/asio/device.rs

Lines 248 to 263 in 11014e4

	pub(crate) fn convert_data_type(ty: &sys::AsioSampleType) -> Option<SampleFormat> {
	let fmt = match *ty {
	sys::AsioSampleType::ASIOSTInt16MSB => SampleFormat::I16,
	sys::AsioSampleType::ASIOSTInt16LSB => SampleFormat::I16,
	sys::AsioSampleType::ASIOSTInt24MSB => SampleFormat::I24,
	sys::AsioSampleType::ASIOSTInt24LSB => SampleFormat::I24,
	sys::AsioSampleType::ASIOSTInt32MSB => SampleFormat::I32,
	sys::AsioSampleType::ASIOSTInt32LSB => SampleFormat::I32,
	sys::AsioSampleType::ASIOSTFloat32MSB => SampleFormat::F32,
	sys::AsioSampleType::ASIOSTFloat32LSB => SampleFormat::F32,
	sys::AsioSampleType::ASIOSTFloat64MSB => SampleFormat::F64,
	sys::AsioSampleType::ASIOSTFloat64LSB => SampleFormat::F64,
	_ => return None,
	};
	Some(fmt)
	}

How would users know which way to serialize their samples?

WASAPI and CoreAudio streams to one device get mixed by the OS. ALSA depends entirely on the PCM: raw hw:X,Y will throw a DeviceBusy error on a second open, dmix and others can allow concurrent access

Ah, so it depends on whether or not there is a mixer. I see my mistake now. In ASIO, all hardware channels are always mono by design, which is why the concept of creating 2 mono streams on a stereo device was distinct from mixing in my head, but I realize now that this is probably a quirk unique to ASIO, and that other APIs would have to interleave the data on the fly, which is effectively just another form of mixing. So what I was thinking of is really just channel mapping again. Nevermind

[roderickvd]

The dream is to eventually replace asio-sys in cpal's bundled ASIO implementation

A permissively licensed, well-functioning Rust ASIO implementation would be amazing! Best of luck and fun while at it.

Samples are typed (f32/i16/etc), not raw bytes, so there's no wire format to swap.

Isn't that precisely what introduces byte order significance? E.g. there is i32::to_le_bytes() and i32::to_be_bytes(). But it seems like cpal's current ASIO implementation doesn't distinguish between them at all:

cpal/src/host/asio/device.rs

Lines 248 to 263 in 11014e4

	pub(crate) fn convert_data_type(ty: &sys::AsioSampleType) -> Option<SampleFormat> {
	let fmt = match *ty {
	sys::AsioSampleType::ASIOSTInt16MSB => SampleFormat::I16,
	sys::AsioSampleType::ASIOSTInt16LSB => SampleFormat::I16,
	sys::AsioSampleType::ASIOSTInt24MSB => SampleFormat::I24,
	sys::AsioSampleType::ASIOSTInt24LSB => SampleFormat::I24,
	sys::AsioSampleType::ASIOSTInt32MSB => SampleFormat::I32,
	sys::AsioSampleType::ASIOSTInt32LSB => SampleFormat::I32,
	sys::AsioSampleType::ASIOSTFloat32MSB => SampleFormat::F32,
	sys::AsioSampleType::ASIOSTFloat32LSB => SampleFormat::F32,
	sys::AsioSampleType::ASIOSTFloat64MSB => SampleFormat::F64,
	sys::AsioSampleType::ASIOSTFloat64LSB => SampleFormat::F64,
	_ => return None,
	};
	Some(fmt)
	}

convert_data_type does collapse ASIOSTInt32LSB/ASIOSTInt32MSB into the same SampleFormat::I32, but that function's only job is validating the requested format against what the driver reports, it's not where conversion happens. The actual byte-swapping lives in stream.rs, which keeps the raw sys::AsioSampleType around and matches on it directly:

(SampleFormat::I32, &sys::AsioSampleType::ASIOSTInt32LSB) => { ... }
(SampleFormat::I32, &sys::AsioSampleType::ASIOSTInt32MSB) => { ... }

Each calling into from_endianness/to_endianness to do the real conversion. So SampleFormat doesn't carry endianness because it's always native.

How would users know which way to serialize their samples?

They wouldn't, and shouldn't. Any wire-format translation is the host implementation's job.

WASAPI and CoreAudio streams to one device get mixed by the OS. ALSA depends entirely on the PCM: raw hw:X,Y will throw a DeviceBusy error on a second open, dmix and others can allow concurrent access

I realize now that this is probably a quirk unique to ASIO, and that other APIs would have to interleave the data on the fly

Correct.

[LastExceed]

that function's only job is validating [...] The actual byte-swapping lives in stream.rs, which keeps the raw sys::AsioSampleType around

Oops, I missed that

Any wire-format translation is the host implementation's job.

Got it, thanks.

One more question: Host::from_host() requires Device to be Clone. This is counter-intuitive to me, as Device represents a physical component of the system (or an instance of an ASIO driver in my case, which is a COM object that also isn't Clone), which obviously cannot be duplicated by software. What is the rationale behind this?

[roderickvd]

One more question: Host::from_host() requires Device to be Clone. This is counter-intuitive to me, as Device represents a physical component of the system (or an instance of an ASIO driver in my case, which is a COM object that also isn't Clone), which obviously cannot be duplicated by software. What is the rationale behind this?

DeviceTrait actually does not have a Clone bound. The Clone requirement only shows up in the custom feature's type-erasure layer, because the public-facing Device/Host wrap a Box<dyn DeviceErased>, and that trait object needs a .clone() because DeviceInner is Clone.

Your intuition that you can't duplicate hardware is understandable but ultimately doesn't hold, because Device is meant as an identifier rather than as a live handle to hardware (and maybe an Arc to a shared context).

[LastExceed]

Device is meant as an identifier rather than as a live handle to hardware

OH. Ok yeah I got that completely wrong. Now I also understand why device enumeration (the Devices iterator) isn't fallible. TYVM