Can we connect a VST instrument to an AudioStream?

[0xdevalias]

A question came up on a discord server I'm in (Ref), as to whether it's possible to connect a VST instrument as the 'input' of an AudioStream; or if not, some other method of allowing to stream the instrument to an output device.

The example for live audio seems to show using both an input/output device, and applying an effects chain:

• https://github.com/spotify/pedalboard#running-pedalboard-on-live-audio

And the docs seem to show that input/output device usage, as well as 'synchronous use', where an ogg_buffer is passed in (but it's not super clear where the output of that goes when .run() is called:

• https://spotify.github.io/pedalboard/reference/pedalboard.io.html#pedalboard.io.AudioStream

Attempting to look at the underlying code didn't help me figure out any better if it was possible:

• pedalboard/pedalboard_native/io/__init__.pyi

  Lines 219 to 323 in 9ab77d3

  	class AudioStream:
  	"""
  	A class that streams audio from an input audio device (i.e.: a microphone,
  	audio interface, etc) to an output device (speaker, headphones),
  	passing it through a :class:`pedalboard.Pedalboard` to add effects.
  	:class:`AudioStream` may be used as a context manager::
  	input_device_name = AudioStream.input_device_names[0]
  	output_device_name = AudioStream.output_device_names[0]
  	with AudioStream(input_device_name, output_device_name) as stream:
  	# In this block, audio is streaming through `stream`!
  	# Audio will be coming out of your speakers at this point.
  	# Add plugins to the live audio stream:
  	reverb = Reverb()
  	stream.plugins.append(reverb)
  	# Change plugin properties as the stream is running:
  	reverb.wet_level = 1.0
  	# Delete plugins:
  	del stream.plugins[0]
  	:class:`AudioStream` may also be used synchronously::
  	stream = AudioStream(ogg_buffer)
  	stream.plugins.append(Reverb(wet_level=1.0))
  	stream.run() # Run the stream until Ctrl-C is received
  	.. note::
  	This class uses C++ under the hood to ensure speed, thread safety,
  	and avoid any locking concerns with Python's Global Interpreter Lock.
  	Audio data processed by :class:`AudioStream` is not available to
  	Python code; the only way to interact with the audio stream is through
  	the :py:attr:`plugins` attribute.
  	.. warning::
  	The :class:`AudioStream` class implements a context manager interface
  	to ensure that audio streams are never left "dangling" (i.e.: running in
  	the background without being stopped).
  	While it is possible to call the :meth:`__enter__` method directly to run an
  	audio stream in the background, this can have some nasty side effects. If the
  	:class:`AudioStream` object is no longer reachable (not bound to a variable,
  	not in scope, etc), the audio stream will continue to play back forever, and
  	won't stop until the Python interpreter exits.
  	To run an :class:`AudioStream` in the background, use Python's
  	:py:mod:`threading` module to call the synchronous :meth:`run` method on a
  	background thread, allowing for easier cleanup.
  	*Introduced in v0.7.0. Not supported on Linux.*
  	"""
  	def __enter__(self) -> AudioStream:
  	"""
  	Use this :class:`AudioStream` as a context manager. Entering the context manager will immediately start the audio stream, sending audio through to the output device.
  	"""
  	def __exit__(self, arg0: object, arg1: object, arg2: object) -> None:
  	"""
  	Exit the context manager, ending the audio stream. Once called, the audio stream will be stopped (i.e.: :py:attr:`running` will be :py:const:`False`).
  	"""
  	def __init__(
  	self,
  	input_device_name: str,
  	output_device_name: str,
  	plugins: typing.Optional[pedalboard_native.utils.Chain] = None,
  	sample_rate: typing.Optional[float] = None,
  	buffer_size: int = 512,
  	allow_feedback: bool = False,
  	) -> None: ...
  	def __repr__(self) -> str: ...
  	def run(self) -> None:
  	"""
  	Start streaming audio from input to output, passing the audio stream through the :py:attr:`plugins` on this AudioStream object. This call will block the current thread until a :py:exc:`KeyboardInterrupt` (``Ctrl-C``) is received.
  	"""
  	@property
  	def plugins(self) -> pedalboard_native.utils.Chain:
  	"""
  	The Pedalboard object that this AudioStream will use to process audio.
  	"""
  	@plugins.setter
  	def plugins(self, arg1: pedalboard_native.utils.Chain) -> None:
  	"""
  	The Pedalboard object that this AudioStream will use to process audio.
  	"""
  	@property
  	def running(self) -> bool:
  	"""
  	:py:const:`True` if this stream is currently streaming live audio from input to output, :py:const:`False` otherwise.
  	"""
  	input_device_names: typing.List[str] = []
  	output_device_names: typing.List[str] = []
  	pass

• pedalboard/pedalboard/python_bindings.cpp

  Line 67 in 9ab77d3

  #include "io/AudioStream.h"

• https://github.com/spotify/pedalboard/blob/9ab77d335e858205a136463735c2154a8ac229bb/pedalboard/io/AudioStream.h

Asking ChatGPT, it suggested using another external library such as the following, and using that on the output from the instrument, to stream it to the desired output device:

• https://github.com/spatialaudio/python-sounddevice
• https://people.csail.mit.edu/hubert/pyaudio/
• https://www.portaudio.com/

See Also

• AudioFile in input of AudioStream #294
• Question: Is it possible to write the output of an AudioStream to a .wav file as it's streaming? #328
• Added an optional audio file parameter to the AudioStream class (issue #264) #317
  • Added an optional audio file parameter to the AudioStream class (issue #264) #317 (comment)
• Question: On demand file playback with effects #343

[0xdevalias]

FYI, #317 was superseded by #363, which will be released with Pedalboard v0.9.12:

• Add buffered playback and recording to AudioStream. #363

[aravart]

To clarify what's meant by this ... Is the issue that VST3Plugin.process when passed with MIDI notes returns the entire duration worth of audio rather than buffer-sized chunks?